2024-02-05T07:16:08.042 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-05T07:16:08.042 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744 --owner scheduled_gabrioux@teuthology --description orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} --name gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi -- /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744/orig.config.yaml 2024-02-05T07:16:08.072 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-05T07:16:08.155 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744 branch: wip-guits-testing-3-2024-02-04-2254 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7546744' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-3-2024-02-04-2254 ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-3-2024-02-04-2254 sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 7317 sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-3-2024-02-04-2254 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOwmtKAcalDVI3B6jBgDB9Ssf8LlUaBhXblimmLIHKW2lqRqQ0VfQPqu9I4AcQXsD3Cc3pKBj2YmYOssbWj+Zkk= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeulAgSUwu4I3ZZhO4QiCggVVW3J4BWvQLe7wktvq7mTzLXwhcqyAMCGQRUb66gY1iNccN8AoH/srT7tkO9Noo= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-05_05:57:01 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-05T07:16:08.155 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa; will attempt to use it 2024-02-05T07:16:08.156 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_164d3be4a527be775f4d3c343ea591122f1b8921/qa/tasks 2024-02-05T07:16:08.156 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-05T07:16:08.157 INFO:teuthology.task.internal:Checking packages... 2024-02-05T07:16:08.186 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '164d3be4a527be775f4d3c343ea591122f1b8921' 2024-02-05T07:16:08.186 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-05T07:16:08.187 INFO:teuthology.packaging:ref: None 2024-02-05T07:16:08.187 INFO:teuthology.packaging:tag: None 2024-02-05T07:16:08.187 INFO:teuthology.packaging:branch: wip-guits-testing-3-2024-02-04-2254 2024-02-05T07:16:08.187 INFO:teuthology.packaging:sha1: 164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:16:08.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-3-2024-02-04-2254 2024-02-05T07:16:08.333 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1241.g164d3be4 2024-02-05T07:16:08.334 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-05T07:16:08.342 INFO:teuthology.task.internal:no buildpackages task found 2024-02-05T07:16:08.342 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-05T07:16:08.348 INFO:teuthology.task.internal:Saving configuration 2024-02-05T07:16:08.358 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-05T07:16:08.365 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T07:16:08.390 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:11:24.208051', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOwmtKAcalDVI3B6jBgDB9Ssf8LlUaBhXblimmLIHKW2lqRqQ0VfQPqu9I4AcQXsD3Cc3pKBj2YmYOssbWj+Zkk='} 2024-02-05T07:16:08.409 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:11:24.206460', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeulAgSUwu4I3ZZhO4QiCggVVW3J4BWvQLe7wktvq7mTzLXwhcqyAMCGQRUb66gY1iNccN8AoH/srT7tkO9Noo='} 2024-02-05T07:16:08.409 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-05T07:16:08.416 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-05T07:16:08.416 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-05T07:16:08.416 INFO:teuthology.run_tasks:Running task console_log... 2024-02-05T07:16:08.482 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc8cd4e20d0>, signals=[15]) 2024-02-05T07:16:08.483 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-05T07:16:08.489 INFO:teuthology.task.internal:Opening connections... 2024-02-05T07:16:08.489 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-02-05T07:16:08.491 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:16:08.596 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-02-05T07:16:08.597 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:16:08.697 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-05T07:16:08.704 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-02-05T07:16:08.721 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-02-05T07:16:08.721 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-02-05T07:16:08.778 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-02-05T07:16:08.779 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-05T07:16:08.779 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-02-05T07:16:08.779 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-05T07:16:08.779 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-05T07:16:08.779 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-05T07:16:08.779 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-02-05T07:16:08.809 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-02-05T07:16:08.827 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-02-05T07:16:08.827 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-05T07:16:08.885 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-05T07:16:08.886 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-05T07:16:08.886 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-02-05T07:16:08.909 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-05T07:16:08.918 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-05T07:16:08.925 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-05T07:16:08.925 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-02-05T07:16:08.928 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-02-05T07:16:08.945 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-05T07:16:08.951 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-05T07:16:08.951 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-02-05T07:16:08.986 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-02-05T07:16:09.007 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-05T07:16:09.061 INFO:teuthology.run_tasks:Running task kernel... 2024-02-05T07:16:09.076 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-05T07:16:09.076 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-05T07:16:09.076 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-05T07:16:09.077 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-05T07:16:09.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:16:09.094 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-02-05T07:16:09.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:16:09.095 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-02-05T07:16:09.151 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-361.el9.x86_64 2024-02-05T07:16:09.152 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.14.0-361.el9.x86_64 2024-02-05T07:16:09.152 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2024-02-05T07:16:09.153 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-361.el9.x86_64 2024-02-05T07:16:09.153 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-361.el9.x86_64 2024-02-05T07:16:09.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-02-05T07:16:11.208 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 14 MB/s | 6.6 MB 00:00 2024-02-05T07:16:11.345 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 11 MB/s | 6.6 MB 00:00 2024-02-05T07:16:12.691 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2024-02-05T07:16:12.870 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.0 MB 00:00 2024-02-05T07:16:14.793 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.0 MB 00:01 2024-02-05T07:16:15.286 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-02-05T07:16:17.683 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-02-05T07:16:18.804 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 19 MB 00:02 2024-02-05T07:16:24.525 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 5.9 MB 00:03 2024-02-05T07:16:26.151 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 15 kB 00:00 2024-02-05T07:16:28.879 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 20 MB 00:02 2024-02-05T07:16:33.985 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-02-05T07:16:36.258 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-05T07:16:36.259 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-05T07:16:36.344 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-416.el9 CentOS-BaseOS 6.0 M 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-416.el9 CentOS-BaseOS 21 M 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-416.el9 CentOS-BaseOS 39 M 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-416.el9 CentOS-BaseOS 33 M 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:16:36.354 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:16:36.355 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:16:36.355 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-02-05T07:16:36.355 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:16:36.355 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 99 M 2024-02-05T07:16:36.355 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 122 M 2024-02-05T07:16:36.355 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:16:37.203 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-416.el9.x86_64.rpm 7.1 MB/s | 6.0 MB 00:00 2024-02-05T07:16:37.878 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-416.el9.x86_64.rpm 14 MB/s | 21 MB 00:01 2024-02-05T07:16:38.578 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 377 kB/s | 5.9 MB 00:16 2024-02-05T07:16:38.770 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-5.14.0-416.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-02-05T07:16:39.253 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-core-5.14.0-416.el9.x86_6 16 MB/s | 33 MB 00:02 2024-02-05T07:16:39.257 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:16:39.257 INFO:teuthology.orchestra.run.smithi043.stdout:Total 34 MB/s | 99 MB 00:02 2024-02-05T07:16:39.257 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:16:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:16:39.706 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:16:40.245 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 15 kB 00:00 2024-02-05T07:16:40.808 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:16:40.809 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:16:43.338 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:16:44.021 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:16:44.154 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:16:47.077 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:16:47.259 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:16:52.354 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:16:52.472 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:16:57.589 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:00.248 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 20 MB 00:19 2024-02-05T07:17:05.312 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-02-05T07:17:07.644 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-05T07:17:07.644 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-05T07:17:07.749 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:17:07.755 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:17:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:17:07.755 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:17:07.755 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:17:07.755 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 5.14.0-416.el9 CentOS-BaseOS 6.0 M 2024-02-05T07:17:07.755 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core x86_64 5.14.0-416.el9 CentOS-BaseOS 21 M 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules x86_64 5.14.0-416.el9 CentOS-BaseOS 39 M 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core x86_64 5.14.0-416.el9 CentOS-BaseOS 33 M 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2024-02-05T07:17:07.756 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:17:07.757 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 99 M 2024-02-05T07:17:07.757 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 122 M 2024-02-05T07:17:07.757 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:17:08.179 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): kernel-5.14.0-416.el9.x86_64.rpm 14 MB/s | 6.0 MB 00:00 2024-02-05T07:17:08.705 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): kernel-core-5.14.0-416.el9.x86_64.rpm 22 MB/s | 21 MB 00:00 2024-02-05T07:17:09.213 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): kernel-modules-core-5.14.0-416.el9.x86_6 32 MB/s | 33 MB 00:01 2024-02-05T07:17:09.755 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): kernel-modules-5.14.0-416.el9.x86_64.rpm 20 MB/s | 39 MB 00:01 2024-02-05T07:17:09.759 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:17:09.759 INFO:teuthology.orchestra.run.smithi050.stdout:Total 50 MB/s | 99 MB 00:02 2024-02-05T07:17:09.759 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:17:10.216 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:17:10.217 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:17:11.255 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:17:11.256 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:17:13.803 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:17:14.589 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-core-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:17:14.723 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:17:17.601 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:17:17.806 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:17:22.896 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:17:23.007 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:28.083 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:28.083 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:28.083 INFO:teuthology.orchestra.run.smithi043.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-05T07:17:28.084 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:17:28.158 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:28.602 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:28.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:17:28.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:17:28.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:17:28.986 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:28.986 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:17:28.986 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:17:28.986 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-416.el9.x86_64 2024-02-05T07:17:28.987 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-416.el9.x86_64 2024-02-05T07:17:28.987 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-416.el9.x86_64 2024-02-05T07:17:28.987 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-416.el9.x86_64 2024-02-05T07:17:28.987 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:17:28.987 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:17:29.487 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:17:29.632 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:17:29.633 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:17:29.634 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-416.el9.x86_64 2024-02-05T07:17:29.634 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-05T07:17:29.634 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-05T07:17:29.634 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-05T07:17:29.634 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-02-05T07:17:29.634 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:17:29.747 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:17:29.748 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:17:29.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-02-05T07:17:29.791 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-02-05T07:17:29.791 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 68.el9 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8264254 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-05T07:17:29.792 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:17:29.793 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-02-05T07:17:29.794 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-05T07:17:29.794 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-416.el9.x86_64 2024-02-05T07:17:29.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-05T07:17:30.415 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-02-05T07:17:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-05T07:17:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-02-05T07:17:31.836 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-02-05T07:17:31.852 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.MmGAx2ztpy 2024-02-05T07:17:31.852 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.MmGAx2ztpy 2024-02-05T07:17:31.923 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.MmGAx2ztpy 2024-02-05T07:17:32.069 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.MmGAx2ztpy is 6KB 2024-02-05T07:17:32.117 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.MmGAx2ztpy 2024-02-05T07:17:32.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-02-05T07:17:32.204 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-05T07:17:32.204 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-05T07:17:32.204 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-05T07:17:32.204 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64.conf 2024-02-05T07:17:32.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64 2024-02-05T07:17:32.365 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-02-05T07:17:58.622 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:58.622 INFO:teuthology.orchestra.run.smithi050.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-05T07:17:58.622 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:17:58.693 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:59.181 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:59.181 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-5.14.0-416.el9.x86_64 1/4 2024-02-05T07:17:59.181 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-core-5.14.0-416.el9.x86_64 2/4 2024-02-05T07:17:59.181 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-5.14.0-416.el9.x86_64 3/4 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-core-5.14.0-416.el9.x86_64 4/4 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-5.14.0-416.el9.x86_64 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core-5.14.0-416.el9.x86_64 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-5.14.0-416.el9.x86_64 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core-5.14.0-416.el9.x86_64 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:17:59.623 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:18:00.157 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:18:00.309 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:18:00.309 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:18:00.310 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-416.el9.x86_64 2024-02-05T07:18:00.310 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-05T07:18:00.310 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-05T07:18:00.310 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-05T07:18:00.310 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2024-02-05T07:18:00.310 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-02-05T07:18:00.423 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-416.el9.x86_64 2024-02-05T07:18:00.423 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-416.el9.x86_64 2024-02-05T07:18:00.423 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-02-05T07:18:00.461 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-02-05T07:18:00.461 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-02-05T07:18:00.461 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-02-05T07:18:00.461 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 68.el9 2024-02-05T07:18:00.461 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-02-05T07:18:00.461 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-05T07:18:00.461 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8264254 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-05T07:18:00.462 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-05T07:18:00.463 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-05T07:18:00.463 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-02-05T07:18:00.463 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:18:00.463 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-02-05T07:18:00.464 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-05T07:18:00.464 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 5.14.0-416.el9.x86_64 2024-02-05T07:18:00.464 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-05T07:18:01.041 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-02-05T07:18:02.369 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-05T07:18:02.370 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-02-05T07:18:02.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:18:02.427 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-05T07:18:02.463 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-02-05T07:18:02.465 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-02-05T07:18:02.481 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.wRnhWzSmr1 2024-02-05T07:18:02.481 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.wRnhWzSmr1 2024-02-05T07:18:02.553 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.wRnhWzSmr1 2024-02-05T07:18:02.693 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.wRnhWzSmr1 is 6KB 2024-02-05T07:18:02.741 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.wRnhWzSmr1 2024-02-05T07:18:02.756 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-02-05T07:18:02.824 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-05T07:18:02.824 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-05T07:18:02.824 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-05T07:18:02.824 INFO:teuthology.orchestra.run.smithi050.stdout:15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64.conf 2024-02-05T07:18:02.825 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-416.el9.x86_64 2024-02-05T07:18:02.969 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-02-05T07:18:20.732 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-02-05T07:18:29.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-02-05T07:18:29.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:18:32.975 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-05T07:18:32.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-05T07:18:32.975 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:18:51.452 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.50 2024-02-05T07:19:00.462 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-05T07:19:00.462 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:19:29.771 DEBUG:teuthology.orchestra.remote:timed out 2024-02-05T07:19:41.780 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-02-05T07:19:41.781 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:19:42.155 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-02-05T07:19:42.543 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-02-05T07:19:42.543 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-416.el9.x86_64"... 2024-02-05T07:19:42.543 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-02-05T07:19:42.557 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-416.el9.x86_64 2024-02-05T07:19:42.558 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-416.el9.x86_64 vs 5.14.0-416.el9.x86_64 2024-02-05T07:19:42.558 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-05T07:19:42.558 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-05T07:19:43.559 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-05T07:19:43.559 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-05T07:19:43.626 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-02-05T07:19:43.661 DEBUG:teuthology.parallel:result is None 2024-02-05T07:20:00.465 DEBUG:teuthology.orchestra.remote:timed out 2024-02-05T07:20:12.466 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-05T07:20:12.467 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:20:12.769 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-02-05T07:20:13.174 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-05T07:20:13.174 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-416.el9.x86_64"... 2024-02-05T07:20:13.175 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-02-05T07:20:13.190 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-416.el9.x86_64 2024-02-05T07:20:13.190 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-416.el9.x86_64 vs 5.14.0-416.el9.x86_64 2024-02-05T07:20:13.190 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-05T07:20:13.190 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-05T07:20:14.191 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-05T07:20:14.191 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-05T07:20:14.268 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2024-02-05T07:20:14.296 DEBUG:teuthology.parallel:result is None 2024-02-05T07:20:14.296 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-05T07:20:14.304 INFO:teuthology.task.internal:Creating test directory... 2024-02-05T07:20:14.304 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-05T07:20:14.307 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-05T07:20:14.323 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-05T07:20:14.330 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-05T07:20:14.336 INFO:teuthology.task.internal:Creating archive directory... 2024-02-05T07:20:14.336 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-05T07:20:14.364 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-05T07:20:14.394 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-05T07:20:14.401 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-05T07:20:14.401 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-05T07:20:14.426 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-05T07:20:14.463 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:20:14.470 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:20:14.478 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:20:14.484 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-05T07:20:14.485 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-05T07:20:14.493 INFO:teuthology.task.internal:Configuring sudo... 2024-02-05T07:20:14.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-05T07:20:14.522 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-05T07:20:14.554 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-05T07:20:14.563 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-05T07:20:14.563 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-05T07:20:14.591 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-05T07:20:14.609 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:20:14.681 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:20:14.732 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:20:14.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:20:14.857 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:20:14.857 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-05T07:20:14.923 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:20:14.965 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-05T07:20:15.021 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:20:15.104 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-05T07:20:15.149 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:20:15.149 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-05T07:20:15.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-02-05T07:20:15.219 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-02-05T07:20:15.262 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T07:20:15.299 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T07:20:15.614 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-05T07:20:15.622 INFO:teuthology.task.internal:Starting timer... 2024-02-05T07:20:15.622 INFO:teuthology.run_tasks:Running task pcp... 2024-02-05T07:20:15.632 INFO:teuthology.run_tasks:Running task selinux... 2024-02-05T07:20:15.639 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-05T07:20:15.640 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-02-05T07:20:15.697 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-02-05T07:20:15.699 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-02-05T07:20:15.757 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-02-05T07:20:15.758 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-05T07:20:15.759 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-02-05T07:20:15.777 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-02-05T07:20:15.777 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-02-05T07:20:15.795 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-02-05T07:20:15.795 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi050.front.sepia.ceph.com': 'permissive'} 2024-02-05T07:20:15.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T07:20:15.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:20:15.844 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T07:20:15.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:20:15.872 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-05T07:20:15.872 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-02-05T07:20:15.921 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-02-05T07:20:15.955 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-05T07:20:15.965 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-05T07:20:15.980 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-05T07:20:15.982 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi050.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-05T07:24:27.448 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com')] 2024-02-05T07:24:27.449 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-02-05T07:24:27.450 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:24:27.541 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-02-05T07:24:27.623 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-02-05T07:24:27.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-05T07:24:27.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-05T07:24:27.712 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-02-05T07:24:27.800 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-02-05T07:24:27.800 INFO:teuthology.run_tasks:Running task clock... 2024-02-05T07:24:27.810 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-05T07:24:27.810 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-05T07:24:27.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T07:24:27.813 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-05T07:24:27.813 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T07:24:27.846 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-05T07:24:27.864 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-05T07:24:27.873 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-05T07:24:27.890 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-05T07:24:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-02-05T07:24:27.909 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-02-05T07:24:27.920 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-02-05T07:24:27.927 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-05T07:24:27.933 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-02-05T07:24:27.943 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-05T07:24:27.950 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-05T07:24:27.966 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-05T07:24:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-02-05T07:24:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T07:24:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-02-05T07:24:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:27.997 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:28.024 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-02-05T07:24:28.028 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T07:24:28.028 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-02-05T07:24:28.028 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:28.028 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:28.028 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:28.028 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T07:24:28.028 INFO:teuthology.run_tasks:Running task pexec... 2024-02-05T07:24:28.039 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-05T07:24:28.039 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi043.front.sepia.ceph.com 2024-02-05T07:24:28.039 DEBUG:teuthology.orchestra.run.smithi043:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-05T07:24:28.040 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi050.front.sepia.ceph.com 2024-02-05T07:24:28.040 DEBUG:teuthology.orchestra.run.smithi050:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-05T07:24:28.635 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:47 ago on Mon 05 Feb 2024 07:23:41 AM UTC. 2024-02-05T07:24:28.663 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:48 ago on Mon 05 Feb 2024 07:23:40 AM UTC. 2024-02-05T07:24:28.791 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:24:28.804 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:24:28.804 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-02-05T07:24:28.804 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:24:28.804 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.1 M 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 9.8 M 2024-02-05T07:24:28.805 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:24:28.822 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:24:28.839 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:24:28.840 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 3.1 M 2024-02-05T07:24:28.840 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 9.8 M 2024-02-05T07:24:28.840 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:24:29.064 INFO:teuthology.orchestra.run.smithi043.stdout:runc-1.1.12-1.el9.x86_64.rpm 12 MB/s | 3.1 MB 00:00 2024-02-05T07:24:29.065 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:24:29.065 INFO:teuthology.orchestra.run.smithi043.stdout:Total 12 MB/s | 3.1 MB 00:00 2024-02-05T07:24:29.065 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:24:29.071 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:24:29.071 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:24:29.099 INFO:teuthology.orchestra.run.smithi050.stdout:runc-1.1.12-1.el9.x86_64.rpm 12 MB/s | 3.1 MB 00:00 2024-02-05T07:24:29.100 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:24:29.100 INFO:teuthology.orchestra.run.smithi050.stdout:Total 12 MB/s | 3.1 MB 00:00 2024-02-05T07:24:29.100 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:24:29.106 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:24:29.106 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:24:29.145 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:24:29.146 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:24:29.179 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:24:29.180 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:24:29.347 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:24:29.396 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:24:29.454 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:24:29.506 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:24:30.006 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:24:30.040 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:24:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:24:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:24:30.341 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:24:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-05T07:24:30.341 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:24:30.341 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:24:30.360 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-05T07:24:30.360 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:24:30.360 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:24:30.360 INFO:teuthology.orchestra.run.smithi050.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-05T07:24:30.360 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:24:30.360 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:24:30.569 DEBUG:teuthology.parallel:result is None 2024-02-05T07:24:30.591 DEBUG:teuthology.parallel:result is None 2024-02-05T07:24:30.591 INFO:teuthology.run_tasks:Running task install... 2024-02-05T07:24:30.600 DEBUG:teuthology.task.install:project ceph 2024-02-05T07:24:30.601 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921'}} 2024-02-05T07:24:30.601 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921'} 2024-02-05T07:24:30.602 INFO:teuthology.task.install:Using flavor: default 2024-02-05T07:24:30.608 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-05T07:24:30.608 INFO:teuthology.task.install:extra packages: [] 2024-02-05T07:24:30.608 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'tag': None, 'wait_for_package': False} 2024-02-05T07:24:30.608 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:24:30.610 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'tag': None, 'wait_for_package': False} 2024-02-05T07:24:30.610 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:24:30.767 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/ 2024-02-05T07:24:30.767 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1241.g164d3be4 2024-02-05T07:24:30.837 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/ 2024-02-05T07:24:30.837 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1241.g164d3be4 2024-02-05T07:24:30.879 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-05T07:24:30.880 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:24:30.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-05T07:24:30.907 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-05T07:24:30.908 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/164d3be4a527be775f4d3c343ea591122f1b8921/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-05T07:24:30.945 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-05T07:24:30.945 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:24:30.946 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-05T07:24:30.973 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-05T07:24:30.973 DEBUG:teuthology.orchestra.run.smithi050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/164d3be4a527be775f4d3c343ea591122f1b8921/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-05T07:24:30.976 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-05T07:24:31.041 DEBUG:teuthology.orchestra.run.smithi050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-05T07:24:31.056 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-05T07:24:31.120 DEBUG:teuthology.orchestra.run.smithi050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-05T07:24:31.121 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-02-05T07:24:31.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-02-05T07:24:31.195 INFO:teuthology.orchestra.run.smithi050.stdout:check_obsoletes = 1 2024-02-05T07:24:31.197 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum clean all 2024-02-05T07:24:31.380 INFO:teuthology.orchestra.run.smithi043.stdout:74 files removed 2024-02-05T07:24:31.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-02-05T07:24:31.444 INFO:teuthology.orchestra.run.smithi050.stdout:74 files removed 2024-02-05T07:24:31.471 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-radosgw 2024-02-05T07:24:32.148 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 192 kB/s | 80 kB 00:00 2024-02-05T07:24:32.265 INFO:teuthology.orchestra.run.smithi050.stdout:ceph packages for x86_64 171 kB/s | 80 kB 00:00 2024-02-05T07:24:32.540 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 29 kB/s | 10 kB 00:00 2024-02-05T07:24:32.649 INFO:teuthology.orchestra.run.smithi050.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-02-05T07:24:32.874 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 6.4 kB/s | 1.9 kB 00:00 2024-02-05T07:24:32.966 INFO:teuthology.orchestra.run.smithi050.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-02-05T07:24:33.318 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 16 MB/s | 6.6 MB 00:00 2024-02-05T07:24:33.494 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - AppStream 13 MB/s | 6.6 MB 00:00 2024-02-05T07:24:34.776 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 6.0 MB/s | 2.0 MB 00:00 2024-02-05T07:24:35.019 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS-9 - BaseOS 5.4 MB/s | 2.0 MB 00:00 2024-02-05T07:24:36.001 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-02-05T07:24:37.043 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.0 MB 00:01 2024-02-05T07:24:37.971 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-02-05T07:24:39.989 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 10 MB/s | 19 MB 00:01 2024-02-05T07:24:43.544 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.9 MB 00:01 2024-02-05T07:24:44.137 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.9 MB 00:00 2024-02-05T07:24:44.979 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 15 kB 00:00 2024-02-05T07:24:46.022 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2024-02-05T07:24:46.948 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 26 MB/s | 20 MB 00:00 2024-02-05T07:24:46.988 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 11 MB/s | 20 MB 00:01 2024-02-05T07:24:51.981 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-02-05T07:24:52.063 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-02-05T07:24:54.704 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:24:54.706 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:24:54.706 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:24:54.706 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:24:54.706 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:24:54.706 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 9.7 M 2024-02-05T07:24:54.706 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrading: 2024-02-05T07:24:54.706 INFO:teuthology.orchestra.run.smithi050.stdout: librados2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.4 M 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 20 M 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 25 k 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 743 k 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-05T07:24:54.707 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 498 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 45 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 131 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 162 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 320 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 298 k 2024-02-05T07:24:54.708 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 99 k 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout:Install 27 Packages 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout:Upgrade 2 Packages 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:24:54.709 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 57 M 2024-02-05T07:24:54.710 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:24:54.720 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 9.7 M 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.4 M 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:24:54.723 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 20 M 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 25 k 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 743 k 2024-02-05T07:24:54.724 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 498 k 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 5.1 M 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-05T07:24:54.725 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 45 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 131 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 162 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 320 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 298 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 99 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-05T07:24:54.726 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout:Install 27 Packages 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 57 M 2024-02-05T07:24:54.727 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:24:55.441 INFO:teuthology.orchestra.run.smithi050.stdout:(1/29): ceph-base-19.0.0-1241.g164d3be4.el9.x86 9.7 MB/s | 5.1 MB 00:00 2024-02-05T07:24:55.491 INFO:teuthology.orchestra.run.smithi050.stdout:(2/29): ceph-selinux-19.0.0-1241.g164d3be4.el9. 505 kB/s | 25 kB 00:00 2024-02-05T07:24:55.575 INFO:teuthology.orchestra.run.smithi050.stdout:(3/29): libcephfs2-19.0.0-1241.g164d3be4.el9.x8 8.7 MB/s | 743 kB 00:00 2024-02-05T07:24:55.675 INFO:teuthology.orchestra.run.smithi050.stdout:(4/29): libradosstriper1-19.0.0-1241.g164d3be4. 4.9 MB/s | 498 kB 00:00 2024-02-05T07:24:56.192 INFO:teuthology.orchestra.run.smithi050.stdout:(5/29): librgw2-19.0.0-1241.g164d3be4.el9.x86_6 9.9 MB/s | 5.1 MB 00:00 2024-02-05T07:24:56.259 INFO:teuthology.orchestra.run.smithi050.stdout:(6/29): python3-ceph-argparse-19.0.0-1241.g164d 674 kB/s | 45 kB 00:00 2024-02-05T07:24:56.335 INFO:teuthology.orchestra.run.smithi050.stdout:(7/29): python3-ceph-common-19.0.0-1241.g164d3b 1.7 MB/s | 131 kB 00:00 2024-02-05T07:24:56.418 INFO:teuthology.orchestra.run.smithi050.stdout:(8/29): python3-cephfs-19.0.0-1241.g164d3be4.el 1.9 MB/s | 162 kB 00:00 2024-02-05T07:24:56.536 INFO:teuthology.orchestra.run.smithi050.stdout:(9/29): ceph-radosgw-19.0.0-1241.g164d3be4.el9. 6.0 MB/s | 9.7 MB 00:01 2024-02-05T07:24:56.569 INFO:teuthology.orchestra.run.smithi050.stdout:(10/29): python3-rados-19.0.0-1241.g164d3be4.el 2.1 MB/s | 320 kB 00:00 2024-02-05T07:24:56.611 INFO:teuthology.orchestra.run.smithi050.stdout:(11/29): python3-rbd-19.0.0-1241.g164d3be4.el9. 3.9 MB/s | 298 kB 00:00 2024-02-05T07:24:56.635 INFO:teuthology.orchestra.run.smithi050.stdout:(12/29): python3-rgw-19.0.0-1241.g164d3be4.el9. 1.4 MB/s | 99 kB 00:00 2024-02-05T07:24:56.797 INFO:teuthology.orchestra.run.smithi043.stdout:(1/29): ceph-base-19.0.0-1241.g164d3be4.el9.x86 3.0 MB/s | 5.1 MB 00:01 2024-02-05T07:24:56.836 INFO:teuthology.orchestra.run.smithi050.stdout:(13/29): boost-program-options-1.75.0-8.el9.x86 469 kB/s | 106 kB 00:00 2024-02-05T07:24:56.861 INFO:teuthology.orchestra.run.smithi050.stdout:(14/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 706 kB/s | 159 kB 00:00 2024-02-05T07:24:56.880 INFO:teuthology.orchestra.run.smithi043.stdout:(2/29): ceph-selinux-19.0.0-1241.g164d3be4.el9. 302 kB/s | 25 kB 00:00 2024-02-05T07:24:56.887 INFO:teuthology.orchestra.run.smithi050.stdout:(15/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 881 kB/s | 44 kB 00:00 2024-02-05T07:24:56.970 INFO:teuthology.orchestra.run.smithi050.stdout:(16/29): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 661 kB 00:00 2024-02-05T07:24:56.996 INFO:teuthology.orchestra.run.smithi050.stdout:(17/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 282 kB 00:00 2024-02-05T07:24:57.021 INFO:teuthology.orchestra.run.smithi050.stdout:(18/29): python3-prettytable-0.7.2-27.el9.noarc 823 kB/s | 41 kB 00:00 2024-02-05T07:24:57.064 INFO:teuthology.orchestra.run.smithi043.stdout:(3/29): ceph-radosgw-19.0.0-1241.g164d3be4.el9. 4.9 MB/s | 9.7 MB 00:01 2024-02-05T07:24:57.123 INFO:teuthology.orchestra.run.smithi043.stdout:(4/29): libcephfs2-19.0.0-1241.g164d3be4.el9.x8 3.0 MB/s | 743 kB 00:00 2024-02-05T07:24:57.163 INFO:teuthology.orchestra.run.smithi050.stdout:(19/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.8 MB/s | 309 kB 00:00 2024-02-05T07:24:57.198 INFO:teuthology.orchestra.run.smithi043.stdout:(5/29): libradosstriper1-19.0.0-1241.g164d3be4. 3.7 MB/s | 498 kB 00:00 2024-02-05T07:24:57.205 INFO:teuthology.orchestra.run.smithi050.stdout:(20/29): libarrow-doc-9.0.0-10.el9.noarch.rpm 623 kB/s | 26 kB 00:00 2024-02-05T07:24:57.247 INFO:teuthology.orchestra.run.smithi050.stdout:(21/29): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-02-05T07:24:57.290 INFO:teuthology.orchestra.run.smithi043.stdout:(6/29): python3-ceph-argparse-19.0.0-1241.g164d 493 kB/s | 45 kB 00:00 2024-02-05T07:24:57.305 INFO:teuthology.orchestra.run.smithi050.stdout:(22/29): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-02-05T07:24:57.331 INFO:teuthology.orchestra.run.smithi050.stdout:(23/29): libunwind-1.6.2-1.el9.x86_64.rpm 804 kB/s | 67 kB 00:00 2024-02-05T07:24:57.373 INFO:teuthology.orchestra.run.smithi043.stdout:(7/29): python3-ceph-common-19.0.0-1241.g164d3b 1.5 MB/s | 131 kB 00:00 2024-02-05T07:24:57.398 INFO:teuthology.orchestra.run.smithi050.stdout:(24/29): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-02-05T07:24:57.423 INFO:teuthology.orchestra.run.smithi050.stdout:(25/29): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-02-05T07:24:57.457 INFO:teuthology.orchestra.run.smithi043.stdout:(8/29): python3-cephfs-19.0.0-1241.g164d3be4.el 1.9 MB/s | 162 kB 00:00 2024-02-05T07:24:57.465 INFO:teuthology.orchestra.run.smithi050.stdout:(26/29): thrift-0.15.0-2.el9.x86_64.rpm 24 MB/s | 1.6 MB 00:00 2024-02-05T07:24:57.557 INFO:teuthology.orchestra.run.smithi043.stdout:(9/29): python3-rados-19.0.0-1241.g164d3be4.el9 3.1 MB/s | 320 kB 00:00 2024-02-05T07:24:57.674 INFO:teuthology.orchestra.run.smithi043.stdout:(10/29): python3-rbd-19.0.0-1241.g164d3be4.el9. 2.5 MB/s | 298 kB 00:00 2024-02-05T07:24:57.800 INFO:teuthology.orchestra.run.smithi043.stdout:(11/29): python3-rgw-19.0.0-1241.g164d3be4.el9. 795 kB/s | 99 kB 00:00 2024-02-05T07:24:57.907 INFO:teuthology.orchestra.run.smithi050.stdout:(27/29): librados2-19.0.0-1241.g164d3be4.el9.x8 7.0 MB/s | 3.4 MB 00:00 2024-02-05T07:24:57.941 INFO:teuthology.orchestra.run.smithi043.stdout:(12/29): boost-program-options-1.75.0-8.el9.x86 749 kB/s | 106 kB 00:00 2024-02-05T07:24:58.000 INFO:teuthology.orchestra.run.smithi043.stdout:(13/29): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 159 kB 00:00 2024-02-05T07:24:58.042 INFO:teuthology.orchestra.run.smithi043.stdout:(14/29): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-05T07:24:58.117 INFO:teuthology.orchestra.run.smithi043.stdout:(15/29): librdkafka-1.6.1-102.el9.x86_64.rpm 8.6 MB/s | 661 kB 00:00 2024-02-05T07:24:58.167 INFO:teuthology.orchestra.run.smithi043.stdout:(16/29): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.5 MB/s | 282 kB 00:00 2024-02-05T07:24:58.209 INFO:teuthology.orchestra.run.smithi043.stdout:(17/29): python3-prettytable-0.7.2-27.el9.noarc 993 kB/s | 41 kB 00:00 2024-02-05T07:24:58.382 INFO:teuthology.orchestra.run.smithi050.stdout:(28/29): librbd1-19.0.0-1241.g164d3be4.el9.x86_ 3.4 MB/s | 3.1 MB 00:00 2024-02-05T07:24:58.418 INFO:teuthology.orchestra.run.smithi043.stdout:(18/29): librgw2-19.0.0-1241.g164d3be4.el9.x86_ 3.9 MB/s | 5.1 MB 00:01 2024-02-05T07:24:58.618 INFO:teuthology.orchestra.run.smithi043.stdout:(19/29): ceph-common-19.0.0-1241.g164d3be4.el9. 5.6 MB/s | 20 MB 00:03 2024-02-05T07:24:58.777 INFO:teuthology.orchestra.run.smithi043.stdout:(20/29): gperftools-libs-2.9.1-2.el9.x86_64.rpm 545 kB/s | 309 kB 00:00 2024-02-05T07:24:59.011 INFO:teuthology.orchestra.run.smithi043.stdout:(21/29): libarrow-doc-9.0.0-10.el9.noarch.rpm 66 kB/s | 26 kB 00:00 2024-02-05T07:24:59.185 INFO:teuthology.orchestra.run.smithi043.stdout:(22/29): liboath-2.6.7-2.el9.x86_64.rpm 119 kB/s | 49 kB 00:00 2024-02-05T07:24:59.652 INFO:teuthology.orchestra.run.smithi043.stdout:(23/29): libunwind-1.6.2-1.el9.x86_64.rpm 105 kB/s | 67 kB 00:00 2024-02-05T07:25:00.869 INFO:teuthology.orchestra.run.smithi043.stdout:(24/29): re2-20211101-3.el9.x86_64.rpm 157 kB/s | 192 kB 00:01 2024-02-05T07:25:02.045 INFO:teuthology.orchestra.run.smithi043.stdout:(25/29): parquet-libs-9.0.0-10.el9.x86_64.rpm 293 kB/s | 838 kB 00:02 2024-02-05T07:25:02.786 INFO:teuthology.orchestra.run.smithi043.stdout:(26/29): librados2-19.0.0-1241.g164d3be4.el9.x8 4.5 MB/s | 3.4 MB 00:00 2024-02-05T07:25:02.903 INFO:teuthology.orchestra.run.smithi043.stdout:(27/29): thrift-0.15.0-2.el9.x86_64.rpm 796 kB/s | 1.6 MB 00:02 2024-02-05T07:25:03.811 INFO:teuthology.orchestra.run.smithi043.stdout:(28/29): librbd1-19.0.0-1241.g164d3be4.el9.x86_ 3.1 MB/s | 3.1 MB 00:01 2024-02-05T07:25:04.341 INFO:teuthology.orchestra.run.smithi050.stdout:(29/29): ceph-common-19.0.0-1241.g164d3be4.el9. 2.1 MB/s | 20 MB 00:09 2024-02-05T07:25:04.345 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:04.346 INFO:teuthology.orchestra.run.smithi050.stdout:Total 5.9 MB/s | 57 MB 00:09 2024-02-05T07:25:04.346 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:25:04.386 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:25:04.386 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:25:04.900 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:25:04.900 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:25:06.249 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:25:06.374 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2024-02-05T07:25:06.412 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/31 2024-02-05T07:25:06.464 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/31 2024-02-05T07:25:06.814 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/31 2024-02-05T07:25:06.870 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:25:06.962 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:25:07.052 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 6/31 2024-02-05T07:25:07.088 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:25:07.144 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:25:07.171 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/31 2024-02-05T07:25:07.299 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 9/31 2024-02-05T07:25:07.369 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:25:07.391 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:25:07.464 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:25:07.575 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2024-02-05T07:25:07.607 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/31 2024-02-05T07:25:07.656 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2024-02-05T07:25:07.696 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2024-02-05T07:25:07.726 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/31 2024-02-05T07:25:07.758 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2024-02-05T07:25:08.083 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2024-02-05T07:25:08.122 INFO:teuthology.orchestra.run.smithi050.stdout: Upgrading : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:25:08.212 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:25:08.243 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 20/31 2024-02-05T07:25:08.284 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2024-02-05T07:25:08.760 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 22/31 2024-02-05T07:25:08.791 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:25:08.879 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:25:08.966 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 24/31 2024-02-05T07:25:08.993 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 25/31 2024-02-05T07:25:09.070 INFO:teuthology.orchestra.run.smithi043.stdout:(29/29): libarrow-9.0.0-10.el9.x86_64.rpm 425 kB/s | 4.4 MB 00:10 2024-02-05T07:25:09.074 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:09.074 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.0 MB/s | 57 MB 00:14 2024-02-05T07:25:09.075 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:25:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:25:09.114 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:25:09.613 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:25:09.613 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:25:10.959 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:25:11.051 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/31 2024-02-05T07:25:11.150 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/31 2024-02-05T07:25:11.299 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:25:11.311 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/31 2024-02-05T07:25:11.332 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:25:11.716 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/31 2024-02-05T07:25:11.753 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:25:11.824 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:25:11.892 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:25:11.916 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 6/31 2024-02-05T07:25:11.944 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:25:11.951 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:25:11.996 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:25:12.026 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/31 2024-02-05T07:25:12.037 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:25:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-05T07:25:12.037 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-05T07:25:12.038 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:12.061 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 9/31 2024-02-05T07:25:12.079 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:25:12.148 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:25:12.169 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:25:12.218 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 11/31 2024-02-05T07:25:12.343 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/31 2024-02-05T07:25:12.377 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/31 2024-02-05T07:25:12.417 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/31 2024-02-05T07:25:12.459 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/31 2024-02-05T07:25:12.492 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/31 2024-02-05T07:25:12.524 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 17/31 2024-02-05T07:25:12.859 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 18/31 2024-02-05T07:25:12.883 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:25:12.973 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 19/31 2024-02-05T07:25:13.004 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 20/31 2024-02-05T07:25:13.051 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 21/31 2024-02-05T07:25:13.529 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 22/31 2024-02-05T07:25:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:25:13.632 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 23/31 2024-02-05T07:25:13.711 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 24/31 2024-02-05T07:25:13.738 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 25/31 2024-02-05T07:25:16.005 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:25:16.042 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:25:16.605 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 26/31 2024-02-05T07:25:16.646 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:25:16.736 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 27/31 2024-02-05T07:25:16.736 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-05T07:25:16.736 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-05T07:25:16.736 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:16.763 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /sys 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /proc 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /mnt 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /var/tmp 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /home 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /root 2024-02-05T07:25:22.105 INFO:teuthology.orchestra.run.smithi050.stdout:skipping the directory /tmp 2024-02-05T07:25:22.106 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:23.039 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:25:23.063 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:25:23.063 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:23.063 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-05T07:25:23.063 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:25:23.063 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:25:23.063 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:23.064 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:25:23.133 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:25:23.133 INFO:teuthology.orchestra.run.smithi050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:25:24.322 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 1/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 3/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 4/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 6/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 8/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 9/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 11/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 12/31 2024-02-05T07:25:24.323 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 13/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 21/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 22/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 25/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2024-02-05T07:25:24.324 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2024-02-05T07:25:24.325 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:25:24.325 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2024-02-05T07:25:24.325 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 30/31 2024-02-05T07:25:25.277 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:25:25.277 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:25.277 INFO:teuthology.orchestra.run.smithi050.stdout:Upgraded: 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-05T07:25:25.278 INFO:teuthology.orchestra.run.smithi050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:25.279 INFO:teuthology.orchestra.run.smithi050.stdout: re2-1:20211101-3.el9.x86_64 2024-02-05T07:25:25.280 INFO:teuthology.orchestra.run.smithi050.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-05T07:25:25.280 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:25.280 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:25:25.614 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-test 2024-02-05T07:25:26.191 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:35 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:25:26.363 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:25:26.363 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 47 M 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 163 k 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:26.364 INFO:teuthology.orchestra.run.smithi050.stdout:Install 6 Packages 2024-02-05T07:25:26.365 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:26.365 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 48 M 2024-02-05T07:25:26.365 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 204 M 2024-02-05T07:25:26.365 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:25:26.566 INFO:teuthology.orchestra.run.smithi050.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-02-05T07:25:26.608 INFO:teuthology.orchestra.run.smithi050.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-02-05T07:25:26.658 INFO:teuthology.orchestra.run.smithi050.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 3.6 MB/s | 186 kB 00:00 2024-02-05T07:25:26.695 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:25:26.695 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-02-05T07:25:26.696 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-02-05T07:25:26.696 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-02-05T07:25:26.696 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-02-05T07:25:26.696 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-02-05T07:25:26.696 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-02-05T07:25:26.696 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-02-05T07:25:26.696 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:26.708 INFO:teuthology.orchestra.run.smithi050.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-02-05T07:25:26.734 INFO:teuthology.orchestra.run.smithi050.stdout:(5/6): libcephsqlite-19.0.0-1241.g164d3be4.el9. 444 kB/s | 163 kB 00:00 2024-02-05T07:25:27.653 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:25:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 29/31 2024-02-05T07:25:27.678 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:27.678 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-05T07:25:27.678 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:25:27.678 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-05T07:25:27.678 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:27.679 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:25:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 30/31 2024-02-05T07:25:27.757 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:25:28.826 INFO:teuthology.orchestra.run.smithi050.stdout:(6/6): ceph-test-19.0.0-1241.g164d3be4.el9.x86_ 19 MB/s | 47 MB 00:02 2024-02-05T07:25:28.829 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:28.829 INFO:teuthology.orchestra.run.smithi050.stdout:Total 20 MB/s | 48 MB 00:02 2024-02-05T07:25:28.829 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:25:28.851 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:25:28.852 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:25:28.861 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:25:28.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 1/31 2024-02-05T07:25:28.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2/31 2024-02-05T07:25:28.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 3/31 2024-02-05T07:25:28.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 4/31 2024-02-05T07:25:28.861 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 5/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_6 6/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 7/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9. 8/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x8 9/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 10/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 11/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 12/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 13/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/31 2024-02-05T07:25:28.862 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/31 2024-02-05T07:25:28.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 20/31 2024-02-05T07:25:28.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 21/31 2024-02-05T07:25:28.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 22/31 2024-02-05T07:25:28.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 23/31 2024-02-05T07:25:28.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 24/31 2024-02-05T07:25:28.863 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 25/31 2024-02-05T07:25:28.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 26/31 2024-02-05T07:25:28.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 27/31 2024-02-05T07:25:28.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 28/31 2024-02-05T07:25:28.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 29/31 2024-02-05T07:25:28.864 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 30/31 2024-02-05T07:25:29.247 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:25:29.248 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 31/31 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.788 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:29.789 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:25:29.841 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:25:29.937 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-05T07:25:29.980 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-05T07:25:30.028 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-05T07:25:30.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-02-05T07:25:30.082 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-05T07:25:30.113 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:25:30.659 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:38 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:25:30.820 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:25:30.821 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:30.821 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:25:30.821 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:30.821 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:25:30.821 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 47 M 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 163 k 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-02-05T07:25:30.822 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:30.823 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 48 M 2024-02-05T07:25:30.823 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 204 M 2024-02-05T07:25:30.823 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:25:31.031 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-02-05T07:25:31.064 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.8 MB/s | 63 kB 00:00 2024-02-05T07:25:31.106 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): jq-1.6-16.el9.x86_64.rpm 4.4 MB/s | 186 kB 00:00 2024-02-05T07:25:31.157 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.2 MB/s | 217 kB 00:00 2024-02-05T07:25:31.198 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): libcephsqlite-19.0.0-1241.g164d3be4.el9. 436 kB/s | 163 kB 00:00 2024-02-05T07:25:33.224 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): ceph-test-19.0.0-1241.g164d3be4.el9.x86_ 20 MB/s | 47 MB 00:02 2024-02-05T07:25:33.227 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:33.227 INFO:teuthology.orchestra.run.smithi043.stdout:Total 20 MB/s | 48 MB 00:02 2024-02-05T07:25:33.227 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:25:33.249 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:25:33.249 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:25:33.652 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:25:33.652 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:25:34.239 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:25:34.321 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-05T07:25:34.366 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-05T07:25:34.409 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-05T07:25:34.467 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-05T07:25:34.493 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:25:34.852 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:25:34.938 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:25:37.429 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:25:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 1/6 2024-02-05T07:25:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2/6 2024-02-05T07:25:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-05T07:25:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-05T07:25:37.430 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-05T07:25:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-05T07:25:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:37.858 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:25:37.858 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: jq-1.6-16.el9.x86_64 2024-02-05T07:25:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-05T07:25:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-05T07:25:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-05T07:25:37.859 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:37.859 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:25:38.135 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph 2024-02-05T07:25:38.714 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:47 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:25:38.881 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout:========================================================================================= 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout:========================================================================================= 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout: ceph x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 6.6 k 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 2.1 M 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 1.5 M 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 247 k 2024-02-05T07:25:38.884 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 4.7 M 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 17 M 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-05T07:25:38.885 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-05T07:25:38.886 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-05T07:25:38.887 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-05T07:25:38.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:38.887 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:25:38.887 INFO:teuthology.orchestra.run.smithi050.stdout:========================================================================================= 2024-02-05T07:25:38.887 INFO:teuthology.orchestra.run.smithi050.stdout:Install 36 Packages 2024-02-05T07:25:38.887 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:38.887 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 30 M 2024-02-05T07:25:38.888 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 107 M 2024-02-05T07:25:38.888 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:25:39.211 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 5/6 2024-02-05T07:25:39.235 INFO:teuthology.orchestra.run.smithi050.stdout:(1/36): ceph-19.0.0-1241.g164d3be4.el9.x86_64.r 41 kB/s | 6.6 kB 00:00 2024-02-05T07:25:39.295 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:25:39.460 INFO:teuthology.orchestra.run.smithi050.stdout:(2/36): ceph-mgr-19.0.0-1241.g164d3be4.el9.x86_ 3.8 MB/s | 1.5 MB 00:00 2024-02-05T07:25:39.527 INFO:teuthology.orchestra.run.smithi050.stdout:(3/36): ceph-mds-19.0.0-1241.g164d3be4.el9.x86_ 4.7 MB/s | 2.1 MB 00:00 2024-02-05T07:25:39.602 INFO:teuthology.orchestra.run.smithi050.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1241.g164d 3.2 MB/s | 247 kB 00:00 2024-02-05T07:25:39.753 INFO:teuthology.orchestra.run.smithi050.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-02-05T07:25:39.803 INFO:teuthology.orchestra.run.smithi050.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-05T07:25:39.870 INFO:teuthology.orchestra.run.smithi050.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 474 kB/s | 32 kB 00:00 2024-02-05T07:25:39.912 INFO:teuthology.orchestra.run.smithi050.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-02-05T07:25:39.954 INFO:teuthology.orchestra.run.smithi050.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 925 kB/s | 39 kB 00:00 2024-02-05T07:25:40.004 INFO:teuthology.orchestra.run.smithi050.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-02-05T07:25:40.096 INFO:teuthology.orchestra.run.smithi050.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-02-05T07:25:40.172 INFO:teuthology.orchestra.run.smithi050.stdout:(12/36): ceph-mon-19.0.0-1241.g164d3be4.el9.x86 5.0 MB/s | 4.7 MB 00:00 2024-02-05T07:25:40.197 INFO:teuthology.orchestra.run.smithi050.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-02-05T07:25:40.222 INFO:teuthology.orchestra.run.smithi050.stdout:(14/36): python3-pycparser-2.20-6.el9.noarch.rp 2.4 MB/s | 124 kB 00:00 2024-02-05T07:25:40.248 INFO:teuthology.orchestra.run.smithi050.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 2.2 MB/s | 113 kB 00:00 2024-02-05T07:25:40.281 INFO:teuthology.orchestra.run.smithi050.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 187 kB 00:00 2024-02-05T07:25:40.490 INFO:teuthology.orchestra.run.smithi050.stdout:(17/36): ceph-osd-19.0.0-1241.g164d3be4.el9.x86 16 MB/s | 17 MB 00:01 2024-02-05T07:25:40.515 INFO:teuthology.orchestra.run.smithi050.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 83 kB/s | 22 kB 00:00 2024-02-05T07:25:40.632 INFO:teuthology.orchestra.run.smithi050.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 431 kB/s | 151 kB 00:00 2024-02-05T07:25:40.649 INFO:teuthology.orchestra.run.smithi050.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 276 kB/s | 43 kB 00:00 2024-02-05T07:25:40.683 INFO:teuthology.orchestra.run.smithi050.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 322 kB/s | 11 kB 00:00 2024-02-05T07:25:40.716 INFO:teuthology.orchestra.run.smithi050.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 529 kB/s | 18 kB 00:00 2024-02-05T07:25:40.742 INFO:teuthology.orchestra.run.smithi050.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.2 MB/s | 358 kB 00:00 2024-02-05T07:25:40.759 INFO:teuthology.orchestra.run.smithi050.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 552 kB/s | 23 kB 00:00 2024-02-05T07:25:40.784 INFO:teuthology.orchestra.run.smithi050.stdout:(25/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 706 kB/s | 172 kB 00:00 2024-02-05T07:25:40.834 INFO:teuthology.orchestra.run.smithi050.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-02-05T07:25:40.859 INFO:teuthology.orchestra.run.smithi050.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 196 kB/s | 20 kB 00:00 2024-02-05T07:25:40.885 INFO:teuthology.orchestra.run.smithi050.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 461 kB/s | 46 kB 00:00 2024-02-05T07:25:40.910 INFO:teuthology.orchestra.run.smithi050.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-02-05T07:25:40.935 INFO:teuthology.orchestra.run.smithi050.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-02-05T07:25:40.960 INFO:teuthology.orchestra.run.smithi050.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-02-05T07:25:40.985 INFO:teuthology.orchestra.run.smithi050.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-02-05T07:25:41.011 INFO:teuthology.orchestra.run.smithi050.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-02-05T07:25:41.036 INFO:teuthology.orchestra.run.smithi050.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-02-05T07:25:41.061 INFO:teuthology.orchestra.run.smithi050.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-02-05T07:25:41.103 INFO:teuthology.orchestra.run.smithi050.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-02-05T07:25:41.108 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:41.109 INFO:teuthology.orchestra.run.smithi050.stdout:Total 13 MB/s | 30 MB 00:02 2024-02-05T07:25:41.126 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:25:41.170 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:25:41.170 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:25:41.591 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:25:41.591 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:25:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 6/6 2024-02-05T07:25:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 1/6 2024-02-05T07:25:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2/6 2024-02-05T07:25:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-05T07:25:41.635 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-05T07:25:41.636 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: jq-1.6-16.el9.x86_64 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-02-05T07:25:42.058 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-02-05T07:25:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-02-05T07:25:42.059 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:42.059 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:25:42.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-02-05T07:25:42.919 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:50 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:25:42.967 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:25:43.031 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-05T07:25:43.081 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-05T07:25:43.096 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout:========================================================================================= 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout:========================================================================================= 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 6.6 k 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 2.1 M 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 1.5 M 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 247 k 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 4.7 M 2024-02-05T07:25:43.099 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 17 M 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-05T07:25:43.100 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-05T07:25:43.101 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-05T07:25:43.101 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-05T07:25:43.103 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-05T07:25:43.104 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-05T07:25:43.104 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:43.104 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:25:43.104 INFO:teuthology.orchestra.run.smithi043.stdout:========================================================================================= 2024-02-05T07:25:43.104 INFO:teuthology.orchestra.run.smithi043.stdout:Install 36 Packages 2024-02-05T07:25:43.104 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:43.105 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 30 M 2024-02-05T07:25:43.105 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 107 M 2024-02-05T07:25:43.105 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:25:43.109 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-05T07:25:43.181 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-05T07:25:43.216 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-05T07:25:43.249 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-05T07:25:43.343 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-05T07:25:43.429 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-05T07:25:43.477 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-05T07:25:43.550 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-05T07:25:43.612 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-05T07:25:43.658 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-05T07:25:43.702 INFO:teuthology.orchestra.run.smithi043.stdout:(1/36): ceph-19.0.0-1241.g164d3be4.el9.x86_64.r 36 kB/s | 6.6 kB 00:00 2024-02-05T07:25:43.707 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-05T07:25:43.780 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-05T07:25:43.903 INFO:teuthology.orchestra.run.smithi043.stdout:(2/36): ceph-mds-19.0.0-1241.g164d3be4.el9.x86_ 5.5 MB/s | 2.1 MB 00:00 2024-02-05T07:25:43.961 INFO:teuthology.orchestra.run.smithi043.stdout:(3/36): ceph-mgr-19.0.0-1241.g164d3be4.el9.x86_ 3.4 MB/s | 1.5 MB 00:00 2024-02-05T07:25:43.969 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-05T07:25:44.013 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-05T07:25:44.036 INFO:teuthology.orchestra.run.smithi043.stdout:(4/36): ceph-mgr-modules-core-19.0.0-1241.g164d 3.2 MB/s | 247 kB 00:00 2024-02-05T07:25:44.051 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-05T07:25:44.094 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-05T07:25:44.132 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-05T07:25:44.186 INFO:teuthology.orchestra.run.smithi043.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-02-05T07:25:44.210 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-05T07:25:44.237 INFO:teuthology.orchestra.run.smithi043.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-02-05T07:25:44.242 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:25:44.276 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-05T07:25:44.279 INFO:teuthology.orchestra.run.smithi043.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 757 kB/s | 32 kB 00:00 2024-02-05T07:25:44.316 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-05T07:25:44.321 INFO:teuthology.orchestra.run.smithi043.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.1 MB/s | 47 kB 00:00 2024-02-05T07:25:44.351 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-05T07:25:44.363 INFO:teuthology.orchestra.run.smithi043.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 928 kB/s | 39 kB 00:00 2024-02-05T07:25:44.386 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-05T07:25:44.455 INFO:teuthology.orchestra.run.smithi043.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.6 MB/s | 241 kB 00:00 2024-02-05T07:25:44.478 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-05T07:25:44.517 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-05T07:25:44.547 INFO:teuthology.orchestra.run.smithi043.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 12 MB/s | 1.1 MB 00:00 2024-02-05T07:25:44.589 INFO:teuthology.orchestra.run.smithi043.stdout:(12/36): python3-ply-3.11-14.el9.noarch.rpm 2.4 MB/s | 103 kB 00:00 2024-02-05T07:25:44.590 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-05T07:25:44.632 INFO:teuthology.orchestra.run.smithi043.stdout:(13/36): python3-pycparser-2.20-6.el9.noarch.rp 2.8 MB/s | 124 kB 00:00 2024-02-05T07:25:44.738 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-05T07:25:44.800 INFO:teuthology.orchestra.run.smithi043.stdout:(14/36): ceph-osd-19.0.0-1241.g164d3be4.el9.x86 19 MB/s | 17 MB 00:00 2024-02-05T07:25:44.825 INFO:teuthology.orchestra.run.smithi043.stdout:(15/36): python3-requests-2.25.1-8.el9.noarch.r 589 kB/s | 113 kB 00:00 2024-02-05T07:25:44.850 INFO:teuthology.orchestra.run.smithi043.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.7 MB/s | 187 kB 00:00 2024-02-05T07:25:44.871 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:25:44.917 INFO:teuthology.orchestra.run.smithi043.stdout:(17/36): ceph-mon-19.0.0-1241.g164d3be4.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-02-05T07:25:45.040 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 31/36 2024-02-05T07:25:45.042 INFO:teuthology.orchestra.run.smithi043.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-02-05T07:25:45.071 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:25:45.095 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:25:45.095 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:45.095 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-05T07:25:45.095 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:25:45.095 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:25:45.095 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:45.276 INFO:teuthology.orchestra.run.smithi043.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-02-05T07:25:45.309 INFO:teuthology.orchestra.run.smithi043.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 111 kB/s | 43 kB 00:00 2024-02-05T07:25:45.418 INFO:teuthology.orchestra.run.smithi043.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 99 kB/s | 11 kB 00:00 2024-02-05T07:25:45.527 INFO:teuthology.orchestra.run.smithi043.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 163 kB/s | 18 kB 00:00 2024-02-05T07:25:45.552 INFO:teuthology.orchestra.run.smithi043.stdout:(23/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-02-05T07:25:45.569 INFO:teuthology.orchestra.run.smithi043.stdout:(24/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 326 kB/s | 172 kB 00:00 2024-02-05T07:25:45.627 INFO:teuthology.orchestra.run.smithi043.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-02-05T07:25:45.653 INFO:teuthology.orchestra.run.smithi043.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 193 kB/s | 19 kB 00:00 2024-02-05T07:25:45.678 INFO:teuthology.orchestra.run.smithi043.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 180 kB/s | 20 kB 00:00 2024-02-05T07:25:45.728 INFO:teuthology.orchestra.run.smithi043.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 460 kB/s | 46 kB 00:00 2024-02-05T07:25:45.829 INFO:teuthology.orchestra.run.smithi043.stdout:(29/36): python3-portend-3.1.0-2.el9.noarch.rpm 164 kB/s | 16 kB 00:00 2024-02-05T07:25:45.845 INFO:teuthology.orchestra.run.smithi043.stdout:(30/36): python3-more-itertools-8.12.0-2.el9.no 410 kB/s | 79 kB 00:00 2024-02-05T07:25:45.870 INFO:teuthology.orchestra.run.smithi043.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-02-05T07:25:45.938 INFO:teuthology.orchestra.run.smithi043.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 827 kB/s | 90 kB 00:00 2024-02-05T07:25:45.963 INFO:teuthology.orchestra.run.smithi043.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 305 kB/s | 36 kB 00:00 2024-02-05T07:25:45.988 INFO:teuthology.orchestra.run.smithi043.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.9 MB/s | 230 kB 00:00 2024-02-05T07:25:46.046 INFO:teuthology.orchestra.run.smithi043.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-02-05T07:25:46.072 INFO:teuthology.orchestra.run.smithi043.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-02-05T07:25:46.076 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:46.076 INFO:teuthology.orchestra.run.smithi043.stdout:Total 10 MB/s | 30 MB 00:02 2024-02-05T07:25:46.096 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:25:46.144 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:25:46.144 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:25:46.582 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:25:46.583 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:25:46.615 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:25:46.643 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:25:46.644 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:46.644 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-05T07:25:46.644 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:25:46.644 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:25:46.644 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:47.132 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:25:47.155 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:25:47.155 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:47.156 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-05T07:25:47.156 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:25:47.156 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:25:47.156 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:47.374 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:25:47.398 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:25:47.398 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:47.398 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-05T07:25:47.398 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:25:47.398 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:25:47.398 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:47.496 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:25:47.950 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:25:48.000 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-02-05T07:25:48.033 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-02-05T07:25:48.061 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-02-05T07:25:48.139 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-02-05T07:25:48.177 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-02-05T07:25:48.212 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-02-05T07:25:48.306 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-02-05T07:25:48.391 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-02-05T07:25:48.442 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-02-05T07:25:48.511 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-02-05T07:25:48.563 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-02-05T07:25:48.601 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-02-05T07:25:48.651 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-02-05T07:25:48.715 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-02-05T07:25:48.907 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-02-05T07:25:48.955 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-02-05T07:25:48.994 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-02-05T07:25:49.028 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-02-05T07:25:49.058 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-02-05T07:25:49.146 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-02-05T07:25:49.177 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:25:49.210 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-02-05T07:25:49.243 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-02-05T07:25:49.286 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-02-05T07:25:49.320 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 1/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 3/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 4/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 5/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 6/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-05T07:25:49.363 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-05T07:25:49.364 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-05T07:25:49.365 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-05T07:25:49.413 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-02-05T07:25:49.451 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-02-05T07:25:49.527 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-02-05T07:25:49.666 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-02-05T07:25:49.799 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:25:49.967 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 31/36 2024-02-05T07:25:49.997 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:25:50.022 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 32/36 2024-02-05T07:25:50.022 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:50.022 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-05T07:25:50.022 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:25:50.022 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-05T07:25:50.022 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:50.313 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-05T07:25:50.313 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:50.313 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: lua-5.4.4-4.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-05T07:25:50.314 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-ply-3.11-14.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-05T07:25:50.315 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:50.316 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:25:50.648 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-base 2024-02-05T07:25:51.221 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:00 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:25:51.329 INFO:teuthology.orchestra.run.smithi050.stdout:Package ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:25:51.383 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:25:51.384 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:25:51.385 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:25:51.446 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install cephadm 2024-02-05T07:25:51.540 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:25:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 33/36 2024-02-05T07:25:51.566 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:51.566 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-05T07:25:51.566 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:25:51.566 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-05T07:25:51.566 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:52.014 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:01 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:25:52.032 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:25:52.054 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 34/36 2024-02-05T07:25:52.054 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:52.054 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-05T07:25:52.055 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:25:52.055 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-05T07:25:52.055 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:52.177 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 748 k 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 748 k 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 754 k 2024-02-05T07:25:52.178 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:25:52.283 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:25:52.305 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 35/36 2024-02-05T07:25:52.305 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:52.306 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-05T07:25:52.306 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:25:52.306 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-05T07:25:52.306 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:52.375 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:25:52.457 INFO:teuthology.orchestra.run.smithi050.stdout:cephadm-19.0.0-1241.g164d3be4.el9.noarch.rpm 2.6 MB/s | 748 kB 00:00 2024-02-05T07:25:52.457 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:52.457 INFO:teuthology.orchestra.run.smithi050.stdout:Total 2.6 MB/s | 748 kB 00:00 2024-02-05T07:25:52.458 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:25:52.463 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:25:52.463 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:25:52.469 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:25:52.469 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:25:52.547 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:25:53.040 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:53.131 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:53.498 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:53.787 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:53.787 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:53.787 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:25:53.788 INFO:teuthology.orchestra.run.smithi050.stdout: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:25:53.788 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:53.788 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:25:53.988 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-immutable-object-cache 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 36/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 1/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 3/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 4/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 5/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9. 6/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-02-05T07:25:54.023 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 16/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 17/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-02-05T07:25:54.024 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-02-05T07:25:54.025 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-02-05T07:25:54.025 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-02-05T07:25:54.025 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-02-05T07:25:54.025 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-02-05T07:25:54.025 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-02-05T07:25:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-02-05T07:25:54.566 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:03 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:25:54.727 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:25:54.728 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:54.728 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-02-05T07:25:54.728 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:54.728 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 145 k 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 145 k 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 436 k 2024-02-05T07:25:54.729 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:25:55.028 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-02-05T07:25:55.028 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:55.028 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:25:55.028 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:55.028 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:55.028 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:55.028 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-05T07:25:55.029 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-05T07:25:55.030 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-05T07:25:55.031 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-05T07:25:55.031 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-05T07:25:55.031 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-05T07:25:55.031 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:55.031 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:25:55.081 INFO:teuthology.orchestra.run.smithi050.stdout:ceph-immutable-object-cache-19.0.0-1241.g164d3b 413 kB/s | 145 kB 00:00 2024-02-05T07:25:55.081 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:55.081 INFO:teuthology.orchestra.run.smithi050.stdout:Total 412 kB/s | 145 kB 00:00 2024-02-05T07:25:55.082 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:25:55.090 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:25:55.090 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:25:55.145 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:25:55.147 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:25:55.297 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:25:55.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-02-05T07:25:55.356 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:25:55.380 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:25:55.381 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:25:55.381 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-05T07:25:55.381 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:55.928 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:03 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:25:56.036 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:25:56.091 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:25:56.092 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:25:56.092 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:25:56.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-02-05T07:25:56.407 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:25:56.407 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:56.407 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:25:56.407 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:25:56.407 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:56.407 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:25:56.657 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr 2024-02-05T07:25:56.731 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:04 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:25:56.893 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:25:56.894 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:56.894 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:25:56.894 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:56.894 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:25:56.894 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 748 k 2024-02-05T07:25:56.894 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:56.895 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:25:56.895 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:56.895 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:25:56.895 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:56.895 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 748 k 2024-02-05T07:25:56.895 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 754 k 2024-02-05T07:25:56.895 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:25:57.238 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:06 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:25:57.332 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-19.0.0-1241.g164d3be4.el9.noarch.rpm 1.7 MB/s | 748 kB 00:00 2024-02-05T07:25:57.333 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:57.333 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.7 MB/s | 748 kB 00:00 2024-02-05T07:25:57.333 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:25:57.338 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:25:57.339 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:25:57.345 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:25:57.345 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:25:57.346 INFO:teuthology.orchestra.run.smithi050.stdout:Package ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:25:57.419 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:25:57.420 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:25:57.420 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:25:57.420 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:25:57.488 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-dashboard 2024-02-05T07:25:57.840 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:57.922 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:58.054 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:07 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:25:58.217 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout:====================================================================================== 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout:====================================================================================== 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 1.8 M 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-05T07:25:58.218 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 28 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 15 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout:====================================================================================== 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout:Install 12 Packages 2024-02-05T07:25:58.219 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:25:58.220 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 6.8 M 2024-02-05T07:25:58.220 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 26 M 2024-02-05T07:25:58.220 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:25:58.297 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:58.535 INFO:teuthology.orchestra.run.smithi050.stdout:(1/12): ceph-prometheus-alerts-19.0.0-1241.g164 75 kB/s | 15 kB 00:00 2024-02-05T07:25:58.560 INFO:teuthology.orchestra.run.smithi050.stdout:(2/12): ceph-grafana-dashboards-19.0.0-1241.g16 124 kB/s | 28 kB 00:00 2024-02-05T07:25:58.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/1 2024-02-05T07:25:58.598 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:58.598 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:25:58.598 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:25:58.598 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:58.599 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:25:58.744 INFO:teuthology.orchestra.run.smithi050.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 238 kB 00:00 2024-02-05T07:25:58.777 INFO:teuthology.orchestra.run.smithi050.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2024-02-05T07:25:58.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-02-05T07:25:58.928 INFO:teuthology.orchestra.run.smithi050.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1241.g164d3be 3.0 MB/s | 1.8 MB 00:00 2024-02-05T07:25:59.061 INFO:teuthology.orchestra.run.smithi050.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 146 kB/s | 19 kB 00:00 2024-02-05T07:25:59.137 INFO:teuthology.orchestra.run.smithi050.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.5 MB/s | 548 kB 00:00 2024-02-05T07:25:59.204 INFO:teuthology.orchestra.run.smithi050.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.1 MB/s | 144 kB 00:00 2024-02-05T07:25:59.262 INFO:teuthology.orchestra.run.smithi050.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 527 kB/s | 31 kB 00:00 2024-02-05T07:25:59.329 INFO:teuthology.orchestra.run.smithi050.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-02-05T07:25:59.364 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:07 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:25:59.421 INFO:teuthology.orchestra.run.smithi050.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.7 MB/s | 2.0 MB 00:00 2024-02-05T07:25:59.527 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 145 k 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:25:59.528 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:25:59.529 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 145 k 2024-02-05T07:25:59.529 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 436 k 2024-02-05T07:25:59.529 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:25:59.563 INFO:teuthology.orchestra.run.smithi050.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.0 MB/s | 863 kB 00:00 2024-02-05T07:25:59.567 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:59.567 INFO:teuthology.orchestra.run.smithi050.stdout:Total 5.1 MB/s | 6.8 MB 00:01 2024-02-05T07:25:59.586 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:25:59.604 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:25:59.604 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:25:59.763 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-19.0.0-1241.g164d3b 623 kB/s | 145 kB 00:00 2024-02-05T07:25:59.763 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:25:59.763 INFO:teuthology.orchestra.run.smithi043.stdout:Total 618 kB/s | 145 kB 00:00 2024-02-05T07:25:59.763 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:25:59.771 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:25:59.771 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:25:59.810 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:25:59.811 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:25:59.825 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:25:59.826 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:25:59.980 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:00.163 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:26:00.187 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:26:00.188 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:26:00.188 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-05T07:26:00.188 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:00.336 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:00.414 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-05T07:26:00.449 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-05T07:26:00.497 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-05T07:26:00.526 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-05T07:26:00.643 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-05T07:26:00.783 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-05T07:26:00.875 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-05T07:26:00.923 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-05T07:26:00.959 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:26:00.989 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 10/12 2024-02-05T07:26:01.244 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 11/12 2024-02-05T07:26:01.274 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:26:01.290 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4. 1/1 2024-02-05T07:26:01.290 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:01.290 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:01.290 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:01.291 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:01.291 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:01.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-02-05T07:26:02.060 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:26:02.060 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 1/12 2024-02-05T07:26:02.060 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 2/12 2024-02-05T07:26:02.060 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 3/12 2024-02-05T07:26:02.060 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-05T07:26:02.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-05T07:26:02.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-05T07:26:02.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-05T07:26:02.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-05T07:26:02.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:26:02.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-05T07:26:02.061 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-05T07:26:02.114 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:10 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:02.223 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:02.280 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:02.281 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:02.281 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:02.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-05T07:26:02.427 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:02.428 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:02.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-05T07:26:02.920 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:10 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:03.083 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 1.8 M 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 28 k 2024-02-05T07:26:03.085 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 15 k 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:03.086 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:03.087 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-02-05T07:26:03.087 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-02-05T07:26:03.087 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:03.087 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 6.8 M 2024-02-05T07:26:03.088 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-02-05T07:26:03.088 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:03.269 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:12 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:03.433 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:03.434 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-02-05T07:26:03.434 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:26:03.434 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-02-05T07:26:03.434 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:03.434 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 7.4 M 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:03.435 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:03.436 INFO:teuthology.orchestra.run.smithi050.stdout:============================================================================================ 2024-02-05T07:26:03.436 INFO:teuthology.orchestra.run.smithi050.stdout:Install 12 Packages 2024-02-05T07:26:03.436 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:03.437 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 37 M 2024-02-05T07:26:03.437 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 209 M 2024-02-05T07:26:03.437 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:03.566 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-grafana-dashboards-19.0.0-1241.g16 146 kB/s | 28 kB 00:00 2024-02-05T07:26:03.591 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): ceph-prometheus-alerts-19.0.0-1241.g164 70 kB/s | 15 kB 00:00 2024-02-05T07:26:03.745 INFO:teuthology.orchestra.run.smithi050.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 209 kB/s | 32 kB 00:00 2024-02-05T07:26:03.775 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 238 kB 00:00 2024-02-05T07:26:03.787 INFO:teuthology.orchestra.run.smithi050.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 427 kB/s | 17 kB 00:00 2024-02-05T07:26:03.826 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-02-05T07:26:03.829 INFO:teuthology.orchestra.run.smithi050.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 838 kB/s | 34 kB 00:00 2024-02-05T07:26:03.879 INFO:teuthology.orchestra.run.smithi050.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-02-05T07:26:03.926 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): ceph-mgr-dashboard-19.0.0-1241.g164d3be 3.2 MB/s | 1.8 MB 00:00 2024-02-05T07:26:04.071 INFO:teuthology.orchestra.run.smithi050.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-02-05T07:26:04.139 INFO:teuthology.orchestra.run.smithi050.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 4.8 MB 00:00 2024-02-05T07:26:04.163 INFO:teuthology.orchestra.run.smithi050.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.4 MB/s | 217 kB 00:00 2024-02-05T07:26:04.210 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 69 kB/s | 19 kB 00:00 2024-02-05T07:26:04.297 INFO:teuthology.orchestra.run.smithi050.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 5.9 MB/s | 802 kB 00:00 2024-02-05T07:26:04.489 INFO:teuthology.orchestra.run.smithi050.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2024-02-05T07:26:04.493 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 821 kB/s | 548 kB 00:00 2024-02-05T07:26:04.514 INFO:teuthology.orchestra.run.smithi050.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 907 kB/s | 194 kB 00:00 2024-02-05T07:26:04.560 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.1 MB/s | 863 kB 00:00 2024-02-05T07:26:04.602 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.3 MB/s | 144 kB 00:00 2024-02-05T07:26:04.739 INFO:teuthology.orchestra.run.smithi050.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 983 kB/s | 245 kB 00:00 2024-02-05T07:26:04.760 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.7 MB/s | 2.0 MB 00:00 2024-02-05T07:26:04.786 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.0 MB/s | 188 kB 00:00 2024-02-05T07:26:04.802 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-repoze-lru-0.7-16.el9.noarch.r 127 kB/s | 31 kB 00:00 2024-02-05T07:26:04.807 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:04.807 INFO:teuthology.orchestra.run.smithi043.stdout:Total 4.0 MB/s | 6.8 MB 00:01 2024-02-05T07:26:04.827 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:05.060 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:05.060 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:05.148 INFO:teuthology.orchestra.run.smithi050.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-1 4.8 MB/s | 7.4 MB 00:01 2024-02-05T07:26:05.150 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:05.151 INFO:teuthology.orchestra.run.smithi050.stdout:Total 22 MB/s | 37 MB 00:01 2024-02-05T07:26:05.164 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:05.189 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:05.189 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:05.568 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:05.568 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:05.575 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:05.646 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-02-05T07:26:05.689 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-02-05T07:26:05.737 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-02-05T07:26:05.774 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-02-05T07:26:05.884 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-02-05T07:26:06.014 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-02-05T07:26:06.139 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-02-05T07:26:06.179 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-02-05T07:26:06.207 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:26:06.237 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 10/12 2024-02-05T07:26:06.292 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:06.364 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-05T07:26:06.394 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-05T07:26:06.495 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 11/12 2024-02-05T07:26:06.499 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:26:06.522 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:26:06.656 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-05T07:26:06.867 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:26:06.905 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:26:06.984 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-05T07:26:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 12/12 2024-02-05T07:26:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el 1/12 2024-02-05T07:26:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noa 2/12 2024-02-05T07:26:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9 3/12 2024-02-05T07:26:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-02-05T07:26:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-02-05T07:26:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-02-05T07:26:07.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-02-05T07:26:07.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-02-05T07:26:07.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-02-05T07:26:07.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-02-05T07:26:07.313 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-02-05T07:26:07.685 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-05T07:26:07.688 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-02-05T07:26:07.689 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-05T07:26:07.690 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-05T07:26:07.690 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:07.690 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:07.796 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-05T07:26:07.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-05T07:26:08.506 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:16 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:08.670 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-02-05T07:26:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:26:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-02-05T07:26:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 7.4 M 2024-02-05T07:26:08.671 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:26:08.671 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-02-05T07:26:08.672 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-02-05T07:26:08.673 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 37 M 2024-02-05T07:26:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 209 M 2024-02-05T07:26:08.673 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:08.876 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 198 kB/s | 32 kB 00:00 2024-02-05T07:26:08.909 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 534 kB/s | 17 kB 00:00 2024-02-05T07:26:08.943 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-02-05T07:26:09.010 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-02-05T07:26:09.116 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-05T07:26:09.219 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2024-02-05T07:26:09.286 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 4.8 MB 00:00 2024-02-05T07:26:09.311 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.3 MB/s | 217 kB 00:00 2024-02-05T07:26:09.545 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 802 kB 00:00 2024-02-05T07:26:09.695 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-02-05T07:26:09.720 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 194 kB 00:00 2024-02-05T07:26:10.154 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-1 5.1 MB/s | 7.4 MB 00:01 2024-02-05T07:26:10.206 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-05T07:26:10.237 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:26:11.195 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 164 kB/s | 245 kB 00:01 2024-02-05T07:26:11.198 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:11.199 INFO:teuthology.orchestra.run.smithi043.stdout:Total 15 MB/s | 37 MB 00:02 2024-02-05T07:26:11.214 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:11.241 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:11.241 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:11.614 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:11.614 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:12.344 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:12.403 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-05T07:26:12.433 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-05T07:26:12.530 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:26:12.570 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-05T07:26:12.755 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:26:12.785 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:26:12.864 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 1/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-05T07:26:12.905 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-05T07:26:13.306 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-05T07:26:13.306 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:13.306 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:13.306 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:13.306 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-05T07:26:13.306 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:13.307 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-05T07:26:13.665 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-rook 2024-02-05T07:26:13.686 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-05T07:26:14.245 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:23 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:14.408 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:14.409 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================== 2024-02-05T07:26:14.409 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:26:14.409 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================== 2024-02-05T07:26:14.409 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 52 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-05T07:26:14.410 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:14.411 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:14.411 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================== 2024-02-05T07:26:14.411 INFO:teuthology.orchestra.run.smithi050.stdout:Install 13 Packages 2024-02-05T07:26:14.411 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:14.411 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 2.0 M 2024-02-05T07:26:14.411 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 26 M 2024-02-05T07:26:14.411 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:15.037 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-05T07:26:15.121 INFO:teuthology.orchestra.run.smithi050.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 113 kB/s | 18 kB 00:00 2024-02-05T07:26:15.146 INFO:teuthology.orchestra.run.smithi050.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 132 kB/s | 24 kB 00:00 2024-02-05T07:26:15.230 INFO:teuthology.orchestra.run.smithi050.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 174 kB 00:00 2024-02-05T07:26:15.255 INFO:teuthology.orchestra.run.smithi050.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 132 kB 00:00 2024-02-05T07:26:15.280 INFO:teuthology.orchestra.run.smithi050.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-02-05T07:26:15.305 INFO:teuthology.orchestra.run.smithi050.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 857 kB/s | 43 kB 00:00 2024-02-05T07:26:15.331 INFO:teuthology.orchestra.run.smithi050.stdout:(7/13): ceph-mgr-rook-19.0.0-1241.g164d3be4.el9 141 kB/s | 52 kB 00:00 2024-02-05T07:26:15.448 INFO:teuthology.orchestra.run.smithi050.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 192 kB/s | 32 kB 00:00 2024-02-05T07:26:15.465 INFO:teuthology.orchestra.run.smithi050.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 94 kB/s | 15 kB 00:00 2024-02-05T07:26:15.532 INFO:teuthology.orchestra.run.smithi050.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 1.3 MB/s | 203 kB 00:00 2024-02-05T07:26:15.549 INFO:teuthology.orchestra.run.smithi050.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-02-05T07:26:15.674 INFO:teuthology.orchestra.run.smithi050.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 634 kB/s | 90 kB 00:00 2024-02-05T07:26:15.708 INFO:teuthology.orchestra.run.smithi050.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.9 MB/s | 1.0 MB 00:00 2024-02-05T07:26:15.712 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:15.712 INFO:teuthology.orchestra.run.smithi050.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2024-02-05T07:26:15.712 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:15.726 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:15.726 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:15.834 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:15.835 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:16.114 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-05T07:26:16.151 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:26:16.376 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:16.553 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-05T07:26:16.610 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-05T07:26:16.666 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-05T07:26:16.702 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-05T07:26:16.744 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-05T07:26:16.841 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-05T07:26:16.968 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-05T07:26:17.020 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-05T07:26:17.788 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-05T07:26:17.842 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-05T07:26:17.876 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-05T07:26:17.934 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-05T07:26:17.970 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 12/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3 1/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-05T07:26:18.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-05T07:26:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-05T07:26:18.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-05T07:26:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:26:18.781 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 1/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-05T07:26:18.782 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-05T07:26:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-05T07:26:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:19.032 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:19.032 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:19.033 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-05T07:26:19.164 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:19.165 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:19.362 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-02-05T07:26:19.452 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-mgr-cephadm 2024-02-05T07:26:19.945 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:27 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:20.027 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:29 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:20.111 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:20.112 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-02-05T07:26:20.112 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:26:20.112 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-02-05T07:26:20.112 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:20.112 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 52 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-05T07:26:20.113 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:20.114 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:20.114 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-02-05T07:26:20.114 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-02-05T07:26:20.114 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:20.114 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.0 M 2024-02-05T07:26:20.114 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-02-05T07:26:20.114 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:20.190 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================== 2024-02-05T07:26:20.191 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:26:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================== 2024-02-05T07:26:20.191 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:20.191 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 138 k 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:======================================================================================== 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout:Install 6 Packages 2024-02-05T07:26:20.192 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 6.8 M 2024-02-05T07:26:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 33 M 2024-02-05T07:26:20.193 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:20.553 INFO:teuthology.orchestra.run.smithi050.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.0 MB/s | 228 kB 00:00 2024-02-05T07:26:20.585 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 181 kB/s | 24 kB 00:00 2024-02-05T07:26:20.594 INFO:teuthology.orchestra.run.smithi050.stdout:(2/6): ceph-mgr-cephadm-19.0.0-1241.g164d3be4.e 537 kB/s | 138 kB 00:00 2024-02-05T07:26:20.602 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 119 kB/s | 18 kB 00:00 2024-02-05T07:26:20.669 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2024-02-05T07:26:20.694 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 132 kB 00:00 2024-02-05T07:26:20.719 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): ceph-mgr-rook-19.0.0-1241.g164d3be4.el9 194 kB/s | 52 kB 00:00 2024-02-05T07:26:20.745 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 211 kB 00:00 2024-02-05T07:26:20.753 INFO:teuthology.orchestra.run.smithi050.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 5.8 MB 00:00 2024-02-05T07:26:20.770 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 569 kB/s | 43 kB 00:00 2024-02-05T07:26:21.012 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 62 kB/s | 15 kB 00:00 2024-02-05T07:26:21.037 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 101 kB/s | 32 kB 00:00 2024-02-05T07:26:21.220 INFO:teuthology.orchestra.run.smithi050.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 97 kB/s | 58 kB 00:00 2024-02-05T07:26:21.237 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 434 kB/s | 203 kB 00:00 2024-02-05T07:26:21.279 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 244 kB/s | 59 kB 00:00 2024-02-05T07:26:21.337 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 897 kB/s | 90 kB 00:00 2024-02-05T07:26:22.862 INFO:teuthology.orchestra.run.smithi050.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 24 kB/s | 51 kB 00:02 2024-02-05T07:26:22.945 INFO:teuthology.orchestra.run.smithi050.stdout:(6/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 229 kB/s | 548 kB 00:02 2024-02-05T07:26:22.949 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:22.949 INFO:teuthology.orchestra.run.smithi050.stdout:Total 2.5 MB/s | 6.8 MB 00:02 2024-02-05T07:26:22.950 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:22.960 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:22.961 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:23.060 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:23.061 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:23.371 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:23.396 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 438 kB/s | 1.0 MB 00:02 2024-02-05T07:26:23.398 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:23.399 INFO:teuthology.orchestra.run.smithi043.stdout:Total 639 kB/s | 2.0 MB 00:03 2024-02-05T07:26:23.399 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:23.415 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:23.530 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:23.533 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:23.553 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-05T07:26:23.598 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-05T07:26:24.096 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:24.159 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-05T07:26:24.246 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-05T07:26:24.277 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-05T07:26:24.315 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2024-02-05T07:26:24.333 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-05T07:26:24.353 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:26:24.388 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-05T07:26:24.433 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-05T07:26:24.474 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-05T07:26:24.576 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-05T07:26:24.690 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-05T07:26:24.751 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-05T07:26:25.207 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:26:25.207 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/6 2024-02-05T07:26:25.207 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-05T07:26:25.208 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2024-02-05T07:26:25.208 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-05T07:26:25.208 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-05T07:26:25.527 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:25.537 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:25.572 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-05T07:26:25.606 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-05T07:26:25.656 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-05T07:26:25.683 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:26:25.782 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-fuse 2024-02-05T07:26:26.357 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:35 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 13/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 1/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-05T07:26:26.521 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-05T07:26:26.522 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-05T07:26:26.523 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 885 k 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:26.524 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:26.525 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:26.525 INFO:teuthology.orchestra.run.smithi050.stdout:Install 2 Packages 2024-02-05T07:26:26.525 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:26.525 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 964 k 2024-02-05T07:26:26.525 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 2.8 M 2024-02-05T07:26:26.525 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:26.682 INFO:teuthology.orchestra.run.smithi050.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 502 kB/s | 79 kB 00:00 2024-02-05T07:26:26.858 INFO:teuthology.orchestra.run.smithi050.stdout:(2/2): ceph-fuse-19.0.0-1241.g164d3be4.el9.x86_ 2.6 MB/s | 885 kB 00:00 2024-02-05T07:26:26.859 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:26.859 INFO:teuthology.orchestra.run.smithi050.stdout:Total 2.8 MB/s | 964 kB 00:00 2024-02-05T07:26:26.859 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:26.881 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:26.881 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-05T07:26:26.894 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-05T07:26:26.895 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-05T07:26:26.896 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:26.896 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:26.960 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:26.960 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:27.128 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-02-05T07:26:27.154 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:27.297 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-05T07:26:27.386 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:26:27.704 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:35 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:27.866 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:27.866 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 138 k 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:27.867 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:27.868 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-02-05T07:26:27.868 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-02-05T07:26:27.868 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:27.868 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 6.8 M 2024-02-05T07:26:27.868 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-02-05T07:26:27.868 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:26:28.103 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/2 2024-02-05T07:26:28.414 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): python3-jinja2-2.11.3-4.el9.noarch.rpm 1.2 MB/s | 228 kB 00:00 2024-02-05T07:26:28.436 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-05T07:26:28.436 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:28.436 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:28.436 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-05T07:26:28.436 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:28.436 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:28.581 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 5.8 MB 00:00 2024-02-05T07:26:28.673 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): ceph-mgr-cephadm-19.0.0-1241.g164d3be4.e 306 kB/s | 138 kB 00:00 2024-02-05T07:26:28.698 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install ceph-volume 2024-02-05T07:26:29.065 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 119 kB/s | 58 kB 00:00 2024-02-05T07:26:29.090 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-asyncssh-2.13.2-3.el9.noarch.rpm 811 kB/s | 548 kB 00:00 2024-02-05T07:26:29.116 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 122 kB/s | 51 kB 00:00 2024-02-05T07:26:29.119 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:29.120 INFO:teuthology.orchestra.run.smithi043.stdout:Total 5.4 MB/s | 6.8 MB 00:01 2024-02-05T07:26:29.120 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:29.130 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:29.130 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:29.224 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:29.225 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:29.287 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:38 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:29.452 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:29.453 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:29.453 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repository Size 2024-02-05T07:26:29.453 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:29.453 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:29.453 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 263 k 2024-02-05T07:26:29.453 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout:Install 2 Packages 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 332 k 2024-02-05T07:26:29.454 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 1.5 M 2024-02-05T07:26:29.455 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:29.524 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:29.613 INFO:teuthology.orchestra.run.smithi050.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 435 kB/s | 69 kB 00:00 2024-02-05T07:26:29.688 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-05T07:26:29.734 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 2/6 2024-02-05T07:26:29.738 INFO:teuthology.orchestra.run.smithi050.stdout:(2/2): ceph-volume-19.0.0-1241.g164d3be4.el9.no 930 kB/s | 263 kB 00:00 2024-02-05T07:26:29.739 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:29.740 INFO:teuthology.orchestra.run.smithi050.stdout:Total 1.1 MB/s | 332 kB 00:00 2024-02-05T07:26:29.740 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:29.750 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:29.750 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:29.793 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:29.793 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:29.948 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:30.129 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-05T07:26:30.165 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:26:30.181 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:26:30.181 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:26:30.181 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-05T07:26:30.181 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:30.281 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-05T07:26:30.363 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-05T07:26:30.424 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 5/6 2024-02-05T07:26:30.454 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:26:30.906 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 1/2 2024-02-05T07:26:31.235 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 6/6 2024-02-05T07:26:31.236 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 1/6 2024-02-05T07:26:31.236 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-05T07:26:31.236 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 3/6 2024-02-05T07:26:31.236 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 4/6 2024-02-05T07:26:31.236 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-05T07:26:31.272 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-05T07:26:31.273 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:31.273 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:31.273 INFO:teuthology.orchestra.run.smithi050.stdout: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:31.273 INFO:teuthology.orchestra.run.smithi050.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-05T07:26:31.273 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:31.273 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-4.el9.noarch 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:31.563 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:31.647 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install librados-devel 2024-02-05T07:26:31.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-02-05T07:26:32.223 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:41 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:32.379 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:40 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:32.389 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:32.390 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:32.390 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-02-05T07:26:32.390 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:32.390 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:32.390 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 127 k 2024-02-05T07:26:32.390 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:32.391 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:32.391 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:32.391 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:26:32.391 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:32.392 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 127 k 2024-02-05T07:26:32.392 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 456 k 2024-02-05T07:26:32.392 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:32.544 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:32.544 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:32.544 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:26:32.544 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:32.544 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:32.544 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 885 k 2024-02-05T07:26:32.544 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:26:32.545 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-05T07:26:32.545 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:32.545 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:32.545 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:32.545 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-02-05T07:26:32.545 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:32.545 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 964 k 2024-02-05T07:26:32.546 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.8 M 2024-02-05T07:26:32.546 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:32.717 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 459 kB/s | 79 kB 00:00 2024-02-05T07:26:33.027 INFO:teuthology.orchestra.run.smithi050.stdout:librados-devel-19.0.0-1241.g164d3be4.el9.x86_64 199 kB/s | 127 kB 00:00 2024-02-05T07:26:33.028 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:33.028 INFO:teuthology.orchestra.run.smithi050.stdout:Total 199 kB/s | 127 kB 00:00 2024-02-05T07:26:33.028 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:33.034 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:33.035 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:33.094 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:33.094 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:33.242 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:33.354 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-fuse-19.0.0-1241.g164d3be4.el9.x86_ 1.0 MB/s | 885 kB 00:00 2024-02-05T07:26:33.377 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:33.377 INFO:teuthology.orchestra.run.smithi043.stdout:Total 1.1 MB/s | 964 kB 00:00 2024-02-05T07:26:33.377 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:33.400 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:33.400 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:33.483 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:33.484 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:33.678 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:33.822 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-05T07:26:33.825 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:33.903 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:26:34.151 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:34.152 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:34.152 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:34.152 INFO:teuthology.orchestra.run.smithi050.stdout: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:34.152 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:34.152 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:34.431 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install libcephfs2 2024-02-05T07:26:34.627 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2/2 2024-02-05T07:26:34.627 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/2 2024-02-05T07:26:34.942 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-05T07:26:34.943 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:34.943 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:34.943 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-05T07:26:34.943 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:34.943 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:35.018 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:44 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:35.126 INFO:teuthology.orchestra.run.smithi050.stdout:Package libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:35.184 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:35.184 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:26:35.184 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:35.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-volume 2024-02-05T07:26:35.266 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install libcephfs-devel 2024-02-05T07:26:35.806 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:35.843 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:44 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:35.969 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:35.969 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:35.969 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-02-05T07:26:35.969 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:35.969 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-1241.g164d3be4.el9 ceph-noarch 263 k 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-02-05T07:26:35.970 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:35.971 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 332 k 2024-02-05T07:26:35.971 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 1.5 M 2024-02-05T07:26:35.971 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:36.007 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:36.007 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 31 k 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 31 k 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 140 k 2024-02-05T07:26:36.008 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:36.104 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 515 kB/s | 69 kB 00:00 2024-02-05T07:26:36.464 INFO:teuthology.orchestra.run.smithi050.stdout:libcephfs-devel-19.0.0-1241.g164d3be4.el9.x86_6 68 kB/s | 31 kB 00:00 2024-02-05T07:26:36.465 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:36.465 INFO:teuthology.orchestra.run.smithi050.stdout:Total 68 kB/s | 31 kB 00:00 2024-02-05T07:26:36.465 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:36.469 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:36.469 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:36.500 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:36.500 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:36.597 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:36.621 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-volume-19.0.0-1241.g164d3be4.el9.no 405 kB/s | 263 kB 00:00 2024-02-05T07:26:36.622 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:36.623 INFO:teuthology.orchestra.run.smithi043.stdout:Total 509 kB/s | 332 kB 00:00 2024-02-05T07:26:36.623 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:36.633 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:36.633 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:36.677 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:36.678 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:36.715 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:36.837 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:37.045 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-02-05T07:26:37.091 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:26:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2/2 2024-02-05T07:26:37.106 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:26:37.106 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-02-05T07:26:37.106 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:37.112 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:37.386 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:37.386 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:37.386 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:37.604 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install librados2 2024-02-05T07:26:37.842 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 1/2 2024-02-05T07:26:38.183 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:47 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:38.210 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-02-05T07:26:38.210 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:38.210 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:38.210 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-1241.g164d3be4.el9.noarch 2024-02-05T07:26:38.210 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-02-05T07:26:38.211 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:38.211 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:38.290 INFO:teuthology.orchestra.run.smithi050.stdout:Package librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:38.346 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:38.347 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:26:38.347 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:38.423 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install librbd1 2024-02-05T07:26:38.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-02-05T07:26:39.001 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:47 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:39.100 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:47 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:39.109 INFO:teuthology.orchestra.run.smithi050.stdout:Package librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:39.164 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:39.165 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:26:39.165 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:39.244 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-rados 2024-02-05T07:26:39.264 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 127 k 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:26:39.265 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:39.266 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 127 k 2024-02-05T07:26:39.266 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 456 k 2024-02-05T07:26:39.266 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:39.602 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-19.0.0-1241.g164d3be4.el9.x86_64 378 kB/s | 127 kB 00:00 2024-02-05T07:26:39.603 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:39.603 INFO:teuthology.orchestra.run.smithi043.stdout:Total 376 kB/s | 127 kB 00:00 2024-02-05T07:26:39.603 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:39.609 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:39.609 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:39.667 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:39.668 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:39.812 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:39.825 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:48 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:39.904 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:39.931 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:39.988 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:39.989 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:26:39.989 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:40.065 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-rgw 2024-02-05T07:26:40.315 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:40.616 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:40.616 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:40.617 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:40.617 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:40.642 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:49 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:40.750 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:40.805 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:40.806 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:26:40.806 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:40.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-02-05T07:26:40.874 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-cephfs 2024-02-05T07:26:41.431 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:49 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:41.450 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:50 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:41.538 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:41.559 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:41.594 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:41.595 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:41.595 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:41.614 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:41.615 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:26:41.615 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:41.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-02-05T07:26:41.694 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install python3-rbd 2024-02-05T07:26:42.231 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:50 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:42.286 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:51 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:42.393 INFO:teuthology.orchestra.run.smithi050.stdout:Package python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:42.394 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 31 k 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:42.395 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:26:42.396 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:42.396 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 31 k 2024-02-05T07:26:42.396 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 140 k 2024-02-05T07:26:42.396 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:42.449 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:42.450 INFO:teuthology.orchestra.run.smithi050.stdout:Nothing to do. 2024-02-05T07:26:42.450 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:42.528 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install rbd-fuse 2024-02-05T07:26:42.641 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-19.0.0-1241.g164d3be4.el9.x86_6 128 kB/s | 31 kB 00:00 2024-02-05T07:26:42.641 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:42.641 INFO:teuthology.orchestra.run.smithi043.stdout:Total 127 kB/s | 31 kB 00:00 2024-02-05T07:26:42.641 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:42.645 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:42.645 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:42.669 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:42.669 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:42.760 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:42.865 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:43.109 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:52 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:43.246 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:43.271 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 85 k 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:43.272 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:43.273 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:26:43.273 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:43.273 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 85 k 2024-02-05T07:26:43.273 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 231 k 2024-02-05T07:26:43.273 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:43.516 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:43.516 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:43.516 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:43.517 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:43.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:43.517 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:43.584 INFO:teuthology.orchestra.run.smithi050.stdout:rbd-fuse-19.0.0-1241.g164d3be4.el9.x86_64.rpm 274 kB/s | 85 kB 00:00 2024-02-05T07:26:43.585 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:43.585 INFO:teuthology.orchestra.run.smithi050.stdout:Total 273 kB/s | 85 kB 00:00 2024-02-05T07:26:43.585 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:43.593 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:43.593 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:43.650 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:43.651 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:43.689 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-02-05T07:26:43.840 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:43.961 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:44.272 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:52 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:44.366 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:44.379 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:44.434 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:44.435 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:44.435 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:44.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-02-05T07:26:44.689 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:44.689 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:44.689 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:44.689 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:44.689 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:44.689 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:44.953 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install rbd-mirror 2024-02-05T07:26:45.075 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:53 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:45.182 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:45.238 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:45.239 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:45.239 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:45.301 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-02-05T07:26:45.533 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:54 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:45.695 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:45.696 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:45.696 INFO:teuthology.orchestra.run.smithi050.stdout: Package Arch Version Repo Size 2024-02-05T07:26:45.696 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 3.1 M 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 13 M 2024-02-05T07:26:45.697 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:45.881 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:53 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:45.987 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:46.042 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:46.043 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:46.043 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:46.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-02-05T07:26:46.686 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:54 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:46.704 INFO:teuthology.orchestra.run.smithi050.stdout:rbd-mirror-19.0.0-1241.g164d3be4.el9.x86_64.rpm 3.1 MB/s | 3.1 MB 00:01 2024-02-05T07:26:46.705 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:46.706 INFO:teuthology.orchestra.run.smithi050.stdout:Total 3.1 MB/s | 3.1 MB 00:01 2024-02-05T07:26:46.706 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:46.718 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:46.718 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:46.794 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:46.798 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:46.798 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:46.850 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:46.851 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:46.851 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:46.915 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-02-05T07:26:47.248 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:47.314 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:47.337 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:47.337 INFO:teuthology.orchestra.run.smithi050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:26:47.337 INFO:teuthology.orchestra.run.smithi050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-05T07:26:47.337 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:26:47.337 INFO:teuthology.orchestra.run.smithi050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:26:47.338 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:47.509 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:55 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:47.618 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:47.674 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:47.675 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:47.675 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:47.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-02-05T07:26:48.336 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:56 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:48.444 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:19.0.0-1241.g164d3be4.el9.x86_64 is already installed. 2024-02-05T07:26:48.474 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:48.474 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:48.474 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:48.474 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:48.474 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:48.474 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:48.498 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:48.500 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-02-05T07:26:48.500 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:48.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-02-05T07:26:48.729 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum -y install rbd-nbd 2024-02-05T07:26:49.146 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:57 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:49.309 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 85 k 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 85 k 2024-02-05T07:26:49.310 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 231 k 2024-02-05T07:26:49.311 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:49.314 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:01:58 ago on Mon 05 Feb 2024 07:24:51 AM UTC. 2024-02-05T07:26:49.478 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-02-05T07:26:49.479 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:49.479 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-02-05T07:26:49.479 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:49.479 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 171 k 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout:Install 1 Package 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 171 k 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 494 k 2024-02-05T07:26:49.480 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-02-05T07:26:49.694 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-19.0.0-1241.g164d3be4.el9.x86_64.rpm 223 kB/s | 85 kB 00:00 2024-02-05T07:26:49.694 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:49.694 INFO:teuthology.orchestra.run.smithi043.stdout:Total 222 kB/s | 85 kB 00:00 2024-02-05T07:26:49.695 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:49.702 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:49.702 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:49.761 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:49.761 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:49.784 INFO:teuthology.orchestra.run.smithi050.stdout:rbd-nbd-19.0.0-1241.g164d3be4.el9.x86_64.rpm 562 kB/s | 171 kB 00:00 2024-02-05T07:26:49.784 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:49.785 INFO:teuthology.orchestra.run.smithi050.stdout:Total 559 kB/s | 171 kB 00:00 2024-02-05T07:26:49.785 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-02-05T07:26:49.794 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-02-05T07:26:49.794 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-02-05T07:26:49.852 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-02-05T07:26:49.853 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-02-05T07:26:49.947 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:50.006 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-02-05T07:26:50.061 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:50.110 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:50.484 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:50.595 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:50.773 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:50.773 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:50.774 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:50.774 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:50.875 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:50.875 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:50.875 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-02-05T07:26:50.875 INFO:teuthology.orchestra.run.smithi050.stdout: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:50.875 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:26:50.875 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-02-05T07:26:50.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-02-05T07:26:51.137 DEBUG:teuthology.parallel:result is None 2024-02-05T07:26:51.580 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:59 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:51.743 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 3.1 M 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:51.744 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:26:51.745 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:51.745 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.1 M 2024-02-05T07:26:51.745 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 13 M 2024-02-05T07:26:51.745 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:52.613 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-19.0.0-1241.g164d3be4.el9.x86_64.rpm 3.6 MB/s | 3.1 MB 00:00 2024-02-05T07:26:52.613 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:52.614 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.6 MB/s | 3.1 MB 00:00 2024-02-05T07:26:52.614 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:52.625 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:52.625 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:52.704 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:52.704 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:53.147 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:53.214 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:53.239 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:53.242 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-05T07:26:53.242 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-05T07:26:53.242 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:26:53.242 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-05T07:26:53.243 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:54.431 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:54.431 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:54.431 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:54.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-02-05T07:26:55.253 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:02:03 ago on Mon 05 Feb 2024 07:24:52 AM UTC. 2024-02-05T07:26:55.429 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-1241.g164d3be4.el9 ceph 171 k 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-02-05T07:26:55.430 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:55.431 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 171 k 2024-02-05T07:26:55.431 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 494 k 2024-02-05T07:26:55.431 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-02-05T07:26:55.642 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-19.0.0-1241.g164d3be4.el9.x86_64.rpm 760 kB/s | 171 kB 00:00 2024-02-05T07:26:55.644 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:26:55.644 INFO:teuthology.orchestra.run.smithi043.stdout:Total 754 kB/s | 171 kB 00:00 2024-02-05T07:26:55.644 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-02-05T07:26:55.651 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-02-05T07:26:55.657 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-02-05T07:26:55.709 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-02-05T07:26:55.710 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-02-05T07:26:55.861 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-02-05T07:26:55.976 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:56.452 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:56.747 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 1/1 2024-02-05T07:26:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:56.748 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-02-05T07:26:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-1241.g164d3be4.el9.x86_64 2024-02-05T07:26:56.748 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:26:56.748 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-02-05T07:26:56.947 DEBUG:teuthology.parallel:result is None 2024-02-05T07:26:56.947 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:26:57.097 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-05T07:26:57.117 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-1241.g164d3be4.el9 2024-02-05T07:26:57.118 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1241.g164d3be4.el9 2024-02-05T07:26:57.118 INFO:teuthology.task.install:The correct ceph version 19.0.0-1241.g164d3be4 is installed. 2024-02-05T07:26:57.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:26:57.260 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-05T07:26:57.285 INFO:teuthology.orchestra.run.smithi050.stdout:19.0.0-1241.g164d3be4.el9 2024-02-05T07:26:57.285 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1241.g164d3be4.el9 2024-02-05T07:26:57.285 INFO:teuthology.task.install:The correct ceph version 19.0.0-1241.g164d3be4 is installed. 2024-02-05T07:26:57.288 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-05T07:26:57.288 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:26:57.288 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-05T07:26:57.316 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:26:57.316 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-05T07:26:57.358 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-05T07:26:57.358 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:26:57.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-02-05T07:26:57.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-05T07:26:57.451 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:26:57.451 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/daemon-helper 2024-02-05T07:26:57.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-05T07:26:57.555 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-05T07:26:57.555 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:26:57.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-05T07:26:57.605 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-05T07:26:57.673 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:26:57.673 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-05T07:26:57.704 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-05T07:26:57.773 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-05T07:26:57.774 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:26:57.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-02-05T07:26:57.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-05T07:26:57.866 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:26:57.866 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/usr/bin/stdin-killer 2024-02-05T07:26:57.896 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-05T07:26:57.965 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-05T07:26:58.065 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'cephadm_mode': 'root'} 2024-02-05T07:26:58.065 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:26:58.066 INFO:tasks.cephadm:Cluster fsid is f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:26:58.066 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-05T07:26:58.066 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.43', 'mon.c': '[v2:172.21.15.43:3301,v1:172.21.15.43:6790]', 'mon.b': '172.21.15.50'} 2024-02-05T07:26:58.066 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-02-05T07:26:58.066 INFO:tasks.cephadm:First mgr is y 2024-02-05T07:26:58.066 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-05T07:26:58.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-02-05T07:26:58.094 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-02-05T07:26:58.124 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-05T07:26:58.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:26:58.265 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '164d3be4a527be775f4d3c343ea591122f1b8921', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76505/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1241-g164d3be4', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1241.g164d3be4'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-05 00:32:51.050045', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-testing-3-2024-02-04-2254', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-05T07:26:58.409 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-guits-testing-3-2024-02-04-2254, sha1 164d3be4a527be775f4d3c343ea591122f1b8921 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:26:58.410 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/x86_64/flavors/default/cephadm 2024-02-05T07:26:58.410 DEBUG:teuthology.orchestra.run.smithi043:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-05T07:27:00.355 INFO:teuthology.orchestra.run.smithi043.stdout:-rw-r--r--. 1 ubuntu ubuntu 766928 Feb 5 07:27 /home/ubuntu/cephtest/cephadm 2024-02-05T07:27:00.356 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-guits-testing-3-2024-02-04-2254/164d3be4a527be775f4d3c343ea591122f1b8921/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-05T07:27:01.417 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 766928 Feb 5 07:27 /home/ubuntu/cephtest/cephadm 2024-02-05T07:27:01.417 DEBUG:teuthology.orchestra.run.smithi043:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-05T07:27:01.435 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-05T07:27:01.466 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 on all hosts... 2024-02-05T07:27:01.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 pull 2024-02-05T07:27:01.478 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 pull 2024-02-05T07:27:02.295 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:27:02.327 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:28:00.699 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-02-05T07:28:00.699 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:28:00.699 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1", 2024-02-05T07:28:00.699 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-02-05T07:28:00.699 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0" 2024-02-05T07:28:00.699 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-02-05T07:28:00.700 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-02-05T07:28:05.493 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-02-05T07:28:05.493 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:28:05.493 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1", 2024-02-05T07:28:05.493 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-02-05T07:28:05.493 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0" 2024-02-05T07:28:05.493 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-02-05T07:28:05.493 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-02-05T07:28:05.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-02-05T07:28:05.556 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-02-05T07:28:05.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-02-05T07:28:05.625 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-02-05T07:28:05.662 INFO:tasks.cephadm:Writing seed config... 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-05T07:28:05.663 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-05T07:28:05.664 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-05T07:28:05.664 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-05T07:28:05.664 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-05T07:28:05.664 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-02-05T07:28:05.664 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-02-05T07:28:05.664 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-05T07:28:05.664 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:28:05.665 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-05T07:28:05.679 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f211dd04-c3f7-11ee-95b6-87774f69a715 mon election default strategy = 3 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-05T07:28:05.680 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service 2024-02-05T07:28:05.722 DEBUG:teuthology.orchestra.run.smithi043:mgr.y> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y.service 2024-02-05T07:28:05.764 INFO:tasks.cephadm:Bootstrapping... 2024-02-05T07:28:05.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 -v bootstrap --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.43 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-05T07:28:05.960 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-02-05T07:28:05.960 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921', '-v', 'bootstrap', '--fsid', 'f211dd04-c3f7-11ee-95b6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.43', '--skip-admin-label'] 2024-02-05T07:28:05.986 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.0 2024-02-05T07:28:05.986 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-05T07:28:05.986 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-02-05T07:28:06.016 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.0 2024-02-05T07:28:06.016 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-02-05T07:28:06.016 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-02-05T07:28:06.024 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-05T07:28:06.024 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-05T07:28:06.030 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-05T07:28:06.030 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-02-05T07:28:06.036 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-02-05T07:28:06.042 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-02-05T07:28:06.042 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-02-05T07:28:06.042 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-02-05T07:28:06.065 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.0 2024-02-05T07:28:06.065 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 4.9.0 is present 2024-02-05T07:28:06.065 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-02-05T07:28:06.065 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-02-05T07:28:06.071 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-05T07:28:06.072 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-05T07:28:06.077 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-05T07:28:06.077 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-02-05T07:28:06.083 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-02-05T07:28:06.089 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-02-05T07:28:06.090 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-02-05T07:28:06.090 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-02-05T07:28:06.090 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:06.090 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 139891227796048 on /run/cephadm/f211dd04-c3f7-11ee-95b6-87774f69a715.lock 2024-02-05T07:28:06.090 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 139891227796048 acquired on /run/cephadm/f211dd04-c3f7-11ee-95b6-87774f69a715.lock 2024-02-05T07:28:06.090 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 3300 ... 2024-02-05T07:28:06.090 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-02-05T07:28:06.091 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:3300, 172.21.15.43:6789], mon addrv is [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-02-05T07:28:06.093 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-02-05T07:28:06.094 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-02-05T07:28:06.096 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-05T07:28:06.096 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-05T07:28:06.096 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-05T07:28:06.098 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-05T07:28:06.098 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-05T07:28:06.098 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-05T07:28:06.098 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-05T07:28:06.098 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-02-05T07:28:06.099 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-05T07:28:06.099 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-02-05T07:28:06.099 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-02-05T07:28:06.099 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-05T07:28:06.100 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-05T07:28:06.100 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:28:06.611 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 2024-02-05T07:28:06.611 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921... 2024-02-05T07:28:06.612 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-02-05T07:28:06.612 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:48ac04de0ee67de328b3ea21477b81735e6e7c0783ebcda067ca4695a8266750 2024-02-05T07:28:06.612 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-05T07:28:06.612 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 2024-02-05T07:28:06.612 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-05T07:28:07.591 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev) 2024-02-05T07:28:07.592 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev) 2024-02-05T07:28:07.592 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-02-05T07:28:08.612 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-02-05T07:28:08.612 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-02-05T07:28:09.627 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQAIjsBltcCPOBAAL/IXyO83kI6kJ61gIXEvEA== 2024-02-05T07:28:10.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQAKjsBlndBJABAASxj9jzZo/+pzg5Lh+xZpKQ== 2024-02-05T07:28:11.513 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQAKjsBlh0c7NhAABe0c7t2lAHwv9eBCmbNvvg== 2024-02-05T07:28:11.514 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-02-05T07:28:12.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-05T07:28:12.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-05T07:28:12.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:12.411 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-05T07:28:12.411 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v2:172.21.15.43:3300,v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-05T07:28:12.411 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-02-05T07:28:12.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:12.412 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-05T07:28:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:28:12.412 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-02-05T07:28:13.482 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-02-05T07:28:13.687 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-05T07:28:13.832 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-05T07:28:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715.target. 2024-02-05T07:28:14.011 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715.target. 2024-02-05T07:28:14.222 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a 2024-02-05T07:28:14.222 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service: Unit ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service not loaded. 2024-02-05T07:28:14.391 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715.target.wants/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@.service. 2024-02-05T07:28:14.747 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:14 smithi043 podman[29350]: 2024-02-05 07:28:14.622630267 +0000 UTC m=+0.114636004 container create c9ed647d8238a19a482b3fcf9daa82c7a4b2af10573fded35f505bd5dbef7220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T07:28:14.956 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:28:14.956 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-02-05T07:28:14.963 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:28:14.963 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-02-05T07:28:14.963 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-02-05T07:28:14.963 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-02-05T07:28:14.963 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-02-05T07:28:14.964 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-02-05T07:28:15.087 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:14 smithi043 podman[29350]: 2024-02-05 07:28:14.833251985 +0000 UTC m=+0.325257682 container init c9ed647d8238a19a482b3fcf9daa82c7a4b2af10573fded35f505bd5dbef7220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T07:28:15.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:14 smithi043 podman[29350]: 2024-02-05 07:28:14.838753106 +0000 UTC m=+0.330758803 container start c9ed647d8238a19a482b3fcf9daa82c7a4b2af10573fded35f505bd5dbef7220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T07:28:15.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:14 smithi043 ceph-mon[29389]: mkfs f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:15.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:14 smithi043 ceph-mon[29389]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T07:28:15.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:14 smithi043 bash[29350]: c9ed647d8238a19a482b3fcf9daa82c7a4b2af10573fded35f505bd5dbef7220 2024-02-05T07:28:15.088 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:14 smithi043 systemd[1]: Started Ceph mon.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:28:16.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:15 smithi043 ceph-mon[29389]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T07:28:16.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:15 smithi043 ceph-mon[29389]: monmap e1: 1 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:28:16.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:15 smithi043 ceph-mon[29389]: fsmap 2024-02-05T07:28:16.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:15 smithi043 ceph-mon[29389]: osdmap e1: 0 total, 0 up, 0 in 2024-02-05T07:28:16.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:15 smithi043 ceph-mon[29389]: mgrmap e1: no daemons active 2024-02-05T07:28:16.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:15 smithi043 ceph-mon[29389]: from='client.? 172.21.15.43:0/751210364' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.546952s) 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-05T07:28:16.062 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.373+0000 7f7ef0188700 1 Processor -- start 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.373+0000 7f7ef0188700 1 -- start start 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.373+0000 7f7ef0188700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.373+0000 7f7ef0188700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee8105390 con 0x7f7ee81049a0 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eedf24700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eedf24700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54732/0 (socket says 172.21.15.43:54732) 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eedf24700 1 -- 172.21.15.43:0/398729296 learned_addr learned my addr 172.21.15.43:0/398729296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eedf24700 1 -- 172.21.15.43:0/398729296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee8105ba0 con 0x7f7ee81049a0 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eedf24700 1 --2- 172.21.15.43:0/398729296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8104dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009e80 tx=0x7f7ed800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7cd05d20b40ee6d server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eecf22700 1 -- 172.21.15.43:0/398729296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed800e070 con 0x7f7ee81049a0 2024-02-05T07:28:16.063 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eecf22700 1 -- 172.21.15.43:0/398729296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed8003a20 con 0x7f7ee81049a0 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.374+0000 7f7eecf22700 1 -- 172.21.15.43:0/398729296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed8003ce0 con 0x7f7ee81049a0 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.375+0000 7f7ef0188700 1 -- 172.21.15.43:0/398729296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 msgr2=0x7f7ee8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.375+0000 7f7ef0188700 1 --2- 172.21.15.43:0/398729296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8104dc0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8009e80 tx=0x7f7ed800b330 comp rx=0 tx=0).stop 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.375+0000 7f7ef0188700 1 -- 172.21.15.43:0/398729296 shutdown_connections 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.375+0000 7f7ef0188700 1 --2- 172.21.15.43:0/398729296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8104dc0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.375+0000 7f7ef0188700 1 -- 172.21.15.43:0/398729296 >> 172.21.15.43:0/398729296 conn(0x7f7ee80fffa0 msgr2=0x7f7ee81023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.375+0000 7f7ef0188700 1 -- 172.21.15.43:0/398729296 shutdown_connections 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.375+0000 7f7ef0188700 1 -- 172.21.15.43:0/398729296 wait complete. 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7ef0188700 1 Processor -- start 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7ef0188700 1 -- start start 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7ef0188700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7ef0188700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee81994e0 con 0x7f7ee81049a0 2024-02-05T07:28:16.064 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7eedf24700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7eedf24700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54734/0 (socket says 172.21.15.43:54734) 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7eedf24700 1 -- 172.21.15.43:0/751210364 learned_addr learned my addr 172.21.15.43:0/751210364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7eedf24700 1 -- 172.21.15.43:0/751210364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed8009b30 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7eedf24700 1 --2- 172.21.15.43:0/751210364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8198fa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8004000 tx=0x7f7ed8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.376+0000 7f7ee6ffd700 1 -- 172.21.15.43:0/751210364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed8004370 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.377+0000 7f7ee6ffd700 1 -- 172.21.15.43:0/751210364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed8004510 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.377+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee81996e0 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.377+0000 7f7ee6ffd700 1 -- 172.21.15.43:0/751210364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7ed80047f0 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.377+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee8199b80 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.377+0000 7f7ee6ffd700 1 -- 172.21.15.43:0/751210364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7ed80127c0 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.377+0000 7f7ee6ffd700 1 -- 172.21.15.43:0/751210364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7ed801dbf0 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.377+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee8192c80 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.380+0000 7f7ee6ffd700 1 -- 172.21.15.43:0/751210364 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7ed802b060 con 0x7f7ee81049a0 2024-02-05T07:28:16.065 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.453+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7ee8106110 con 0x7f7ee81049a0 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.453+0000 7f7ee6ffd700 1 -- 172.21.15.43:0/751210364 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f7ed8021030 con 0x7f7ee81049a0 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.454+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 msgr2=0x7f7ee8198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.454+0000 7f7ef0188700 1 --2- 172.21.15.43:0/751210364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8198fa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7ed8004000 tx=0x7f7ed8004ad0 comp rx=0 tx=0).stop 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.454+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 shutdown_connections 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.454+0000 7f7ef0188700 1 --2- 172.21.15.43:0/751210364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ee81049a0 0x7f7ee8198fa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.454+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 >> 172.21.15.43:0/751210364 conn(0x7f7ee80fffa0 msgr2=0x7f7ee818fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.454+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 shutdown_connections 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:15.454+0000 7f7ef0188700 1 -- 172.21.15.43:0/751210364 wait complete. 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-02-05T07:28:16.066 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-02-05T07:28:17.174 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:16 smithi043 ceph-mon[29389]: from='client.? 172.21.15.43:0/4141581190' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-05T07:28:17.174 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:16 smithi043 ceph-mon[29389]: from='client.? 172.21.15.43:0/4141581190' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-05T07:28:17.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:17.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-02-05T07:28:17.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:17.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.43:3300,v1:172.21.15.43:6789] 2024-02-05T07:28:17.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-05T07:28:17.182 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.538+0000 7fe7864cf700 1 Processor -- start 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.538+0000 7fe7864cf700 1 -- start start 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.538+0000 7fe7864cf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780104db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.538+0000 7fe7864cf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe780105380 con 0x7fe780104990 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.538+0000 7fe77ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:17.183 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.538+0000 7fe77ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780104db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54746/0 (socket says 172.21.15.43:54746) 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.538+0000 7fe77ffff700 1 -- 172.21.15.43:0/3508262159 learned_addr learned my addr 172.21.15.43:0/3508262159 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe77ffff700 1 -- 172.21.15.43:0/3508262159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe780105b90 con 0x7fe780104990 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe77ffff700 1 --2- 172.21.15.43:0/3508262159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780104db0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098a0 tx=0x7fe770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2887685f01b990e server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe77effd700 1 -- 172.21.15.43:0/3508262159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe770004160 con 0x7fe780104990 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe77effd700 1 -- 172.21.15.43:0/3508262159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7700042c0 con 0x7fe780104990 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe77effd700 1 -- 172.21.15.43:0/3508262159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe7700045a0 con 0x7fe780104990 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe7864cf700 1 -- 172.21.15.43:0/3508262159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 msgr2=0x7fe780104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe7864cf700 1 --2- 172.21.15.43:0/3508262159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780104db0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098a0 tx=0x7fe770009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.539+0000 7fe7864cf700 1 -- 172.21.15.43:0/3508262159 shutdown_connections 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.540+0000 7fe7864cf700 1 --2- 172.21.15.43:0/3508262159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780104db0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.540+0000 7fe7864cf700 1 -- 172.21.15.43:0/3508262159 >> 172.21.15.43:0/3508262159 conn(0x7fe7800fffd0 msgr2=0x7fe7801023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.540+0000 7fe7864cf700 1 -- 172.21.15.43:0/3508262159 shutdown_connections 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.540+0000 7fe7864cf700 1 -- 172.21.15.43:0/3508262159 wait complete. 2024-02-05T07:28:17.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.540+0000 7fe7864cf700 1 Processor -- start 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.540+0000 7fe7864cf700 1 -- start start 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe7864cf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe7864cf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7801983b0 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe77ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe77ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54762/0 (socket says 172.21.15.43:54762) 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe77ffff700 1 -- 172.21.15.43:0/4141581190 learned_addr learned my addr 172.21.15.43:0/4141581190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe77ffff700 1 -- 172.21.15.43:0/4141581190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe770009550 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe77ffff700 1 --2- 172.21.15.43:0/4141581190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780197e70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe770004000 tx=0x7fe770004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.541+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe770011630 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.542+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe770011790 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.542+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7801985b0 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.542+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe770011a30 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.542+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe780198a50 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.542+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe770022070 con 0x7fe780104990 2024-02-05T07:28:17.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.542+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe77001bdb0 con 0x7fe780104990 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.543+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe780191ab0 con 0x7fe780104990 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.545+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe77001b450 con 0x7fe780104990 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.617+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fe7801060e0 con 0x7fe780104990 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.619+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+338 (secure 0 0 0) 0x7fe77001b5f0 con 0x7fe780104990 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.620+0000 7fe77d7fa700 1 -- 172.21.15.43:0/4141581190 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fe77001b790 con 0x7fe780104990 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.621+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 msgr2=0x7fe780197e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.621+0000 7fe7864cf700 1 --2- 172.21.15.43:0/4141581190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780197e70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe770004000 tx=0x7fe770004750 comp rx=0 tx=0).stop 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.621+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 shutdown_connections 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.621+0000 7fe7864cf700 1 --2- 172.21.15.43:0/4141581190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe780104990 0x7fe780197e70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.621+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 >> 172.21.15.43:0/4141581190 conn(0x7fe7800fffd0 msgr2=0x7fe78018ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.621+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 shutdown_connections 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:16.621+0000 7fe7864cf700 1 -- 172.21.15.43:0/4141581190 wait complete. 2024-02-05T07:28:17.186 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-02-05T07:28:18.247 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:17 smithi043 ceph-mon[29389]: from='client.? 172.21.15.43:0/3292096340' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:28:18.249 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6dd925700 1 Processor -- start 2024-02-05T07:28:18.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6dd925700 1 -- start start 2024-02-05T07:28:18.254 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6dd925700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8108970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6dd925700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80796c0 con 0x7fa6d8106580 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6d6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8108970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6d6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8108970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54778/0 (socket says 172.21.15.43:54778) 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6d6ffd700 1 -- 172.21.15.43:0/926286738 learned_addr learned my addr 172.21.15.43:0/926286738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.610+0000 7fa6d6ffd700 1 -- 172.21.15.43:0/926286738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d8108eb0 con 0x7fa6d8106580 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6d6ffd700 1 --2- 172.21.15.43:0/926286738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8108970 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80098a0 tx=0x7fa6c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dac306c50c0bbd server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6d5ffb700 1 -- 172.21.15.43:0/926286738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6c8004160 con 0x7fa6d8106580 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6d5ffb700 1 -- 172.21.15.43:0/926286738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa6c80042c0 con 0x7fa6d8106580 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6d5ffb700 1 -- 172.21.15.43:0/926286738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6c80118a0 con 0x7fa6d8106580 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6dd925700 1 -- 172.21.15.43:0/926286738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 msgr2=0x7fa6d8108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6dd925700 1 --2- 172.21.15.43:0/926286738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8108970 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80098a0 tx=0x7fa6c8009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6dd925700 1 -- 172.21.15.43:0/926286738 shutdown_connections 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6dd925700 1 --2- 172.21.15.43:0/926286738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8108970 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6dd925700 1 -- 172.21.15.43:0/926286738 >> 172.21.15.43:0/926286738 conn(0x7fa6d80fff70 msgr2=0x7fa6d8102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:18.255 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6dd925700 1 -- 172.21.15.43:0/926286738 shutdown_connections 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.611+0000 7fa6dd925700 1 -- 172.21.15.43:0/926286738 wait complete. 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.612+0000 7fa6dd925700 1 Processor -- start 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.612+0000 7fa6dd925700 1 -- start start 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.612+0000 7fa6dd925700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8105db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.612+0000 7fa6dd925700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d81062f0 con 0x7fa6d8106580 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.612+0000 7fa6d6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8105db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.612+0000 7fa6d6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8105db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54782/0 (socket says 172.21.15.43:54782) 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.612+0000 7fa6d6ffd700 1 -- 172.21.15.43:0/3292096340 learned_addr learned my addr 172.21.15.43:0/3292096340 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6d6ffd700 1 -- 172.21.15.43:0/3292096340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c8009550 con 0x7fa6d8106580 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6d6ffd700 1 --2- 172.21.15.43:0/3292096340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8105db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8003da0 tx=0x7fa6c8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6dc923700 1 -- 172.21.15.43:0/3292096340 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6c8011930 con 0x7fa6d8106580 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6dc923700 1 -- 172.21.15.43:0/3292096340 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa6c8003b00 con 0x7fa6d8106580 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d8104530 con 0x7fa6d8106580 2024-02-05T07:28:18.256 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6dc923700 1 -- 172.21.15.43:0/3292096340 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6c801ad50 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d81049d0 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6dc923700 1 -- 172.21.15.43:0/3292096340 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa6c802d7d0 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.613+0000 7fa6dc923700 1 -- 172.21.15.43:0/3292096340 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa6c8024730 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.614+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d8063350 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.616+0000 7fa6dc923700 1 -- 172.21.15.43:0/3292096340 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa6c8019db0 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.686+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fa6d8109db0 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.687+0000 7fa6dc923700 1 -- 172.21.15.43:0/3292096340 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7fa6c8035090 con 0x7fa6d8106580 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.687+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 msgr2=0x7fa6d8105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.687+0000 7fa6dd925700 1 --2- 172.21.15.43:0/3292096340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8105db0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8003da0 tx=0x7fa6c8011e90 comp rx=0 tx=0).stop 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.687+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 shutdown_connections 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.687+0000 7fa6dd925700 1 --2- 172.21.15.43:0/3292096340 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6d8106580 0x7fa6d8105db0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.687+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 >> 172.21.15.43:0/3292096340 conn(0x7fa6d80fff70 msgr2=0x7fa6d8100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.687+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 shutdown_connections 2024-02-05T07:28:18.257 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:17.688+0000 7fa6dd925700 1 -- 172.21.15.43:0/3292096340 wait complete. 2024-02-05T07:28:18.258 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-02-05T07:28:18.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:18 smithi043 systemd[1]: Stopping Ceph mon.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:28:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:18 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a[29366]: 2024-02-05T07:28:18.517+0000 7fc483edb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T07:28:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:18 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a[29366]: 2024-02-05T07:28:18.517+0000 7fc483edb700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-05T07:28:18.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:18 smithi043 podman[29721]: 2024-02-05 07:28:18.64151273 +0000 UTC m=+0.231021978 container died c9ed647d8238a19a482b3fcf9daa82c7a4b2af10573fded35f505bd5dbef7220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-05T07:28:19.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:18 smithi043 podman[29721]: 2024-02-05 07:28:18.981740649 +0000 UTC m=+0.571249901 container cleanup c9ed647d8238a19a482b3fcf9daa82c7a4b2af10573fded35f505bd5dbef7220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-05T07:28:19.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:18 smithi043 bash[29721]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a 2024-02-05T07:28:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 podman[29741]: 2024-02-05 07:28:19.240996128 +0000 UTC m=+0.604655027 container remove c9ed647d8238a19a482b3fcf9daa82c7a4b2af10573fded35f505bd5dbef7220 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-05T07:28:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-05T07:28:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 systemd[1]: Stopped Ceph mon.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:28:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 systemd[1]: Starting Ceph mon.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:28:19.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 podman[29841]: 2024-02-05 07:28:19.517172057 +0000 UTC m=+0.017538959 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:28:19.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 podman[29841]: 2024-02-05 07:28:19.625428959 +0000 UTC m=+0.125795883 container create cd9e9007486b18bb0d10503fc893f0f528fc49aa015a9df29590f10def18d65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-05T07:28:19.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 podman[29841]: 2024-02-05 07:28:19.819384654 +0000 UTC m=+0.319751537 container init cd9e9007486b18bb0d10503fc893f0f528fc49aa015a9df29590f10def18d65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-05T07:28:19.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 podman[29841]: 2024-02-05 07:28:19.824584912 +0000 UTC m=+0.324951794 container start cd9e9007486b18bb0d10503fc893f0f528fc49aa015a9df29590f10def18d65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: pidfile_write: ignore empty --pid-file 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: load: jerasure load: lrc 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: RocksDB version: 7.9.2 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Git sha 0 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: DB SUMMARY 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: DB Session ID: BZHXAWIVO3WVN65WH6DF 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: CURRENT file: CURRENT 2024-02-05T07:28:19.922 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 86815 ; 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.error_if_exists: 0 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.create_if_missing: 0 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.paranoid_checks: 1 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.env: 0x557af670dae0 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.info_log: 0x557af815d8c0 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.statistics: (nil) 2024-02-05T07:28:19.923 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.use_fsync: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_log_file_size: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.allow_fallocate: 1 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.use_direct_reads: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T07:28:19.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.db_log_dir: 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.wal_dir: 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.write_buffer_manager: 0x557af8131400 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T07:28:19.925 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.unordered_write: 0 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.row_cache: None 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.wal_filter: None 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.two_write_queues: 0 2024-02-05T07:28:19.926 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.wal_compression: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.atomic_flush: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.log_readahead_size: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_background_jobs: 2 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_background_compactions: -1 2024-02-05T07:28:19.927 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_subcompactions: 1 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_open_files: -1 2024-02-05T07:28:19.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T07:28:19.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T07:28:19.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_background_flushes: -1 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Compression algorithms supported: 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kZSTD supported: 0 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kXpressCompression supported: 0 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kZlibCompression supported: 1 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kSnappyCompression supported: 1 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kLZ4Compression supported: 1 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: kBZip2Compression supported: 0 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-05T07:28:19.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.merge_operator: 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_filter: None 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557af815d9c0) 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-02-05T07:28:19.931 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x557af8161090 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-02-05T07:28:19.932 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-02-05T07:28:19.933 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression: NoCompression 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.num_levels: 7 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T07:28:19.934 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T07:28:19.935 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T07:28:19.936 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T07:28:19.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.table_properties_collectors: 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.inplace_update_support: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.bloom_locality: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.max_successive_merges: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T07:28:19.938 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.ttl: 2592000 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.enable_blob_files: false 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.min_blob_size: 0 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-05T07:28:19.939 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f631825e-d96a-404c-b1d8-34f24810ce87 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118099854008, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118099854893, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 83237, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 262, "table_properties": {"data_size": 81409, "index_size": 238, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10592, "raw_average_key_size": 47, "raw_value_size": 75421, "raw_average_value_size": 341, "num_data_blocks": 10, "num_entries": 221, "num_filter_entries": 221, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707118099, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f631825e-d96a-404c-b1d8-34f24810ce87", "db_session_id": "BZHXAWIVO3WVN65WH6DF", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118099854984, "job": 1, "event": "recovery_finished"} 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557af8252000 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: DB pointer 0x557af823c000 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T07:28:19.940 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 83.12 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 83.12 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 134.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T07:28:19.941 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 134.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 27.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 27.33 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x557af8161090#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-02-05T07:28:19.942 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: starting mon.a rank 0 at public addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] at bind addrs [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???) e1 preinit fsid f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???).mds e1 new map 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???).mds e1 print_map 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 ceph-mon[29879]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-05T07:28:19.943 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 bash[29841]: cd9e9007486b18bb0d10503fc893f0f528fc49aa015a9df29590f10def18d65a 2024-02-05T07:28:19.944 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:19 smithi043 systemd[1]: Started Ceph mon.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:28:19.944 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-05T07:28:20.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:20 smithi043 ceph-mon[29879]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-05T07:28:20.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:20 smithi043 ceph-mon[29879]: monmap e1: 1 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:28:20.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:20 smithi043 ceph-mon[29879]: fsmap 2024-02-05T07:28:20.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:20 smithi043 ceph-mon[29879]: osdmap e1: 0 total, 0 up, 0 in 2024-02-05T07:28:20.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:20 smithi043 ceph-mon[29879]: mgrmap e1: no daemons active 2024-02-05T07:28:20.968 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4158982435' entity='client.admin' 2024-02-05T07:28:20.970 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.311+0000 7f75cbdc7700 1 Processor -- start 2024-02-05T07:28:20.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.311+0000 7f75cbdc7700 1 -- start start 2024-02-05T07:28:20.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.311+0000 7f75cbdc7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:20.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.311+0000 7f75cbdc7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c4106470 con 0x7f75c4105a80 2024-02-05T07:28:20.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.311+0000 7f75c9b63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.311+0000 7f75c9b63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54788/0 (socket says 172.21.15.43:54788) 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.311+0000 7f75c9b63700 1 -- 172.21.15.43:0/3166804197 learned_addr learned my addr 172.21.15.43:0/3166804197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.312+0000 7f75c9b63700 1 -- 172.21.15.43:0/3166804197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c4106c80 con 0x7f75c4105a80 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.312+0000 7f75c9b63700 1 --2- 172.21.15.43:0/3166804197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4105ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75c00098a0 tx=0x7f75c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b9f11dfc225f7b server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.312+0000 7f75c8b61700 1 -- 172.21.15.43:0/3166804197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75c0004160 con 0x7f75c4105a80 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.312+0000 7f75c8b61700 1 -- 172.21.15.43:0/3166804197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f75c00042c0 con 0x7f75c4105a80 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.312+0000 7f75c8b61700 1 -- 172.21.15.43:0/3166804197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75c00118a0 con 0x7f75c4105a80 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 -- 172.21.15.43:0/3166804197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 msgr2=0x7f75c4105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 --2- 172.21.15.43:0/3166804197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4105ea0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f75c00098a0 tx=0x7f75c0009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 -- 172.21.15.43:0/3166804197 shutdown_connections 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 --2- 172.21.15.43:0/3166804197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4105ea0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:20.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 -- 172.21.15.43:0/3166804197 >> 172.21.15.43:0/3166804197 conn(0x7f75c4101040 msgr2=0x7f75c4103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 -- 172.21.15.43:0/3166804197 shutdown_connections 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 -- 172.21.15.43:0/3166804197 wait complete. 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 Processor -- start 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.313+0000 7f75cbdc7700 1 -- start start 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75cbdc7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75cbdc7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c4199c60 con 0x7f75c4105a80 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75c9b63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75c9b63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54794/0 (socket says 172.21.15.43:54794) 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75c9b63700 1 -- 172.21.15.43:0/4158982435 learned_addr learned my addr 172.21.15.43:0/4158982435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75c9b63700 1 -- 172.21.15.43:0/4158982435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c0009550 con 0x7f75c4105a80 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75c9b63700 1 --2- 172.21.15.43:0/4158982435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4199720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f75c0003da0 tx=0x7f75c0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75baffd700 1 -- 172.21.15.43:0/4158982435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75c0011930 con 0x7f75c4105a80 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75baffd700 1 -- 172.21.15.43:0/4158982435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f75c0003b00 con 0x7f75c4105a80 2024-02-05T07:28:20.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c4199e60 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75baffd700 1 -- 172.21.15.43:0/4158982435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f75c001ad50 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.314+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c419a300 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.315+0000 7f75baffd700 1 -- 172.21.15.43:0/4158982435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f75c002d7d0 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.315+0000 7f75baffd700 1 -- 172.21.15.43:0/4158982435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f75c0019700 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.315+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c4063350 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.318+0000 7f75baffd700 1 -- 172.21.15.43:0/4158982435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f75c001a630 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.388+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f75c419ccd0 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.389+0000 7f75baffd700 1 -- 172.21.15.43:0/4158982435 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f75c002dc10 con 0x7f75c4105a80 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.390+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 msgr2=0x7f75c4199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.390+0000 7f75cbdc7700 1 --2- 172.21.15.43:0/4158982435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4199720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f75c0003da0 tx=0x7f75c0011e90 comp rx=0 tx=0).stop 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.390+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 shutdown_connections 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.390+0000 7f75cbdc7700 1 --2- 172.21.15.43:0/4158982435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f75c4105a80 0x7f75c4199720 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.390+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 >> 172.21.15.43:0/4158982435 conn(0x7f75c4101040 msgr2=0x7f75c4101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:20.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.390+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 shutdown_connections 2024-02-05T07:28:20.979 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:20.390+0000 7f75cbdc7700 1 -- 172.21.15.43:0/4158982435 wait complete. 2024-02-05T07:28:20.979 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-05T07:28:21.001 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-05T07:28:21.001 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-02-05T07:28:21.001 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-05T07:28:21.002 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-05T07:28:21.257 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y 2024-02-05T07:28:21.257 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y.service: Unit ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y.service not loaded. 2024-02-05T07:28:21.410 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715.target.wants/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@.service. 2024-02-05T07:28:21.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:21 smithi043 systemd[1]: Starting Ceph mgr.y for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:28:21.905 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:21 smithi043 podman[30121]: 2024-02-05 07:28:21.627440842 +0000 UTC m=+0.103463156 container create 240db77c263ddc893bc4550df38b329f8ca6731d256f8a43729e24c525edb8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-02-05T07:28:21.906 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:21 smithi043 podman[30121]: 2024-02-05 07:28:21.541669882 +0000 UTC m=+0.017692161 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:28:21.906 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:21 smithi043 podman[30121]: 2024-02-05 07:28:21.821491606 +0000 UTC m=+0.297513878 container init 240db77c263ddc893bc4550df38b329f8ca6731d256f8a43729e24c525edb8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y, ceph=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T07:28:21.906 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:21 smithi043 podman[30121]: 2024-02-05 07:28:21.826894191 +0000 UTC m=+0.302916465 container start 240db77c263ddc893bc4550df38b329f8ca6731d256f8a43729e24c525edb8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T07:28:21.906 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:21 smithi043 bash[30121]: 240db77c263ddc893bc4550df38b329f8ca6731d256f8a43729e24c525edb8c4 2024-02-05T07:28:21.921 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:28:21.922 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-02-05T07:28:21.930 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:28:21.930 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-02-05T07:28:21.930 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-02-05T07:28:21.930 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-02-05T07:28:21.938 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:28:21.938 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-02-05T07:28:21.945 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:28:21.945 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-02-05T07:28:21.946 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-02-05T07:28:21.946 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-05T07:28:21.946 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-02-05T07:28:21.946 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-02-05T07:28:22.173 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:21 smithi043 systemd[1]: Started Ceph mgr.y for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:28:22.476 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:22.394+0000 7fec9125d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:28:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4011956970' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:28:22.814 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:22.493+0000 7fec9125d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:28:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:28:23.041 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "f211dd04-c3f7-11ee-95b6-87774f69a715", 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:28:23.042 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:28:23.043 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:28:23.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:28:23.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:28:23.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:23.052 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:28:14.908485+0000", 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.394+0000 7fa02eb6f700 1 Processor -- start 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.394+0000 7fa02eb6f700 1 -- start start 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.394+0000 7fa02eb6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.394+0000 7fa02eb6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028078e50 con 0x7fa028106540 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.394+0000 7fa02c90b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.394+0000 7fa02c90b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42388/0 (socket says 172.21.15.43:42388) 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.394+0000 7fa02c90b700 1 -- 172.21.15.43:0/2257394885 learned_addr learned my addr 172.21.15.43:0/2257394885 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.395+0000 7fa02c90b700 1 -- 172.21.15.43:0/2257394885 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028108e70 con 0x7fa028106540 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.395+0000 7fa02c90b700 1 --2- 172.21.15.43:0/2257394885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028108930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0098a0 tx=0x7fa01c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266e8826ab9fb959 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:23.053 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.395+0000 7fa0277fe700 1 -- 172.21.15.43:0/2257394885 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa01c004160 con 0x7fa028106540 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.395+0000 7fa0277fe700 1 -- 172.21.15.43:0/2257394885 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa01c0042c0 con 0x7fa028106540 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.395+0000 7fa0277fe700 1 -- 172.21.15.43:0/2257394885 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa01c0118a0 con 0x7fa028106540 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.395+0000 7fa02eb6f700 1 -- 172.21.15.43:0/2257394885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 msgr2=0x7fa028108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.395+0000 7fa02eb6f700 1 --2- 172.21.15.43:0/2257394885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028108930 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa01c0098a0 tx=0x7fa01c009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 -- 172.21.15.43:0/2257394885 shutdown_connections 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 --2- 172.21.15.43:0/2257394885 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028108930 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 -- 172.21.15.43:0/2257394885 >> 172.21.15.43:0/2257394885 conn(0x7fa0280fffd0 msgr2=0x7fa0281023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 -- 172.21.15.43:0/2257394885 shutdown_connections 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 -- 172.21.15.43:0/2257394885 wait complete. 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 Processor -- start 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 -- start start 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.396+0000 7fa02eb6f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa028199c80 con 0x7fa028106540 2024-02-05T07:28:23.054 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa02c90b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa02c90b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42394/0 (socket says 172.21.15.43:42394) 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa02c90b700 1 -- 172.21.15.43:0/4011956970 learned_addr learned my addr 172.21.15.43:0/4011956970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa02c90b700 1 -- 172.21.15.43:0/4011956970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c009550 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa02c90b700 1 --2- 172.21.15.43:0/4011956970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028199740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa01c011890 tx=0x7fa01c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa025ffb700 1 -- 172.21.15.43:0/4011956970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa01c0036a0 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa028199e80 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa025ffb700 1 -- 172.21.15.43:0/4011956970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa01c01a430 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa025ffb700 1 -- 172.21.15.43:0/4011956970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa01c0193c0 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.397+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02819a320 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.398+0000 7fa025ffb700 1 -- 172.21.15.43:0/4011956970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa01c0199d0 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.398+0000 7fa025ffb700 1 -- 172.21.15.43:0/4011956970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa01c00b720 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.398+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa028063350 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.401+0000 7fa025ffb700 1 -- 172.21.15.43:0/4011956970 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa01c02a060 con 0x7fa028106540 2024-02-05T07:28:23.055 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.474+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa028109dc0 con 0x7fa028106540 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.475+0000 7fa025ffb700 1 -- 172.21.15.43:0/4011956970 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fa01c01a5a0 con 0x7fa028106540 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.476+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 msgr2=0x7fa028199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.476+0000 7fa02eb6f700 1 --2- 172.21.15.43:0/4011956970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028199740 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa01c011890 tx=0x7fa01c011970 comp rx=0 tx=0).stop 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.476+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 shutdown_connections 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.476+0000 7fa02eb6f700 1 --2- 172.21.15.43:0/4011956970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa028106540 0x7fa028199740 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.476+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 >> 172.21.15.43:0/4011956970 conn(0x7fa0280fffd0 msgr2=0x7fa028100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.476+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 shutdown_connections 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:22.476+0000 7fa02eb6f700 1 -- 172.21.15.43:0/4011956970 wait complete. 2024-02-05T07:28:23.056 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-02-05T07:28:23.505 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:23.164+0000 7fec9125d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:28:23.505 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:23.262+0000 7fec9125d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:28:23.505 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:23.405+0000 7fec9125d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:28:23.506 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:23.503+0000 7fec9125d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:28:23.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:23.588+0000 7fec9125d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:28:24.159 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:23.857+0000 7fec9125d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:28:24.159 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:23.972+0000 7fec9125d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:28:24.159 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:24 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:24.060+0000 7fec9125d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:28:24.159 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:24 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:24.157+0000 7fec9125d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:28:24.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:24 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:24.334+0000 7fec9125d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:28:25.468 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:25 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:25.343+0000 7fec9125d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:28:25.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:25 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:25.479+0000 7fec9125d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:28:26.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3681618206' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:28:26.116 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:25 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:25.823+0000 7fec9125d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:28:26.116 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:25 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:25.916+0000 7fec9125d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:28:26.117 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:26 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:26.114+0000 7fec9125d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:28:26.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:26.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:28:26.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "f211dd04-c3f7-11ee-95b6-87774f69a715", 2024-02-05T07:28:26.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:28:26.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:28:26.179 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:28:26.180 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:26.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:28:26.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:26.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:28:26.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:28:26.181 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:28:26.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:28:26.190 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:28:14.908485+0000", 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf1a731700 1 Processor -- start 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf1a731700 1 -- start start 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf1a731700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf14105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf1a731700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf14106490 con 0x7faf14105aa0 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf13fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf14105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:26.191 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf13fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf14105ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42408/0 (socket says 172.21.15.43:42408) 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf13fff700 1 -- 172.21.15.43:0/3084116066 learned_addr learned my addr 172.21.15.43:0/3084116066 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.486+0000 7faf13fff700 1 -- 172.21.15.43:0/3084116066 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf14106ca0 con 0x7faf14105aa0 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.487+0000 7faf13fff700 1 --2- 172.21.15.43:0/3084116066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf14105ec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf040098a0 tx=0x7faf04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7957b368742a1a6b server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.487+0000 7faf12ffd700 1 -- 172.21.15.43:0/3084116066 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf04004160 con 0x7faf14105aa0 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.487+0000 7faf12ffd700 1 -- 172.21.15.43:0/3084116066 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7faf040042c0 con 0x7faf14105aa0 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.487+0000 7faf12ffd700 1 -- 172.21.15.43:0/3084116066 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf040118a0 con 0x7faf14105aa0 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 -- 172.21.15.43:0/3084116066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 msgr2=0x7faf14105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 --2- 172.21.15.43:0/3084116066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf14105ec0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7faf040098a0 tx=0x7faf04009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 -- 172.21.15.43:0/3084116066 shutdown_connections 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 --2- 172.21.15.43:0/3084116066 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf14105ec0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 -- 172.21.15.43:0/3084116066 >> 172.21.15.43:0/3084116066 conn(0x7faf14101080 msgr2=0x7faf141034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 -- 172.21.15.43:0/3084116066 shutdown_connections 2024-02-05T07:28:26.192 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 -- 172.21.15.43:0/3084116066 wait complete. 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 Processor -- start 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.488+0000 7faf1a731700 1 -- start start 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf1a731700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf1407ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf1a731700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1407b040 con 0x7faf14105aa0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf13fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf1407ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf13fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf1407ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42420/0 (socket says 172.21.15.43:42420) 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf13fff700 1 -- 172.21.15.43:0/3681618206 learned_addr learned my addr 172.21.15.43:0/3681618206 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf13fff700 1 -- 172.21.15.43:0/3681618206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04009550 con 0x7faf14105aa0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf13fff700 1 --2- 172.21.15.43:0/3681618206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf1407ab00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf04003da0 tx=0x7faf04011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf117fa700 1 -- 172.21.15.43:0/3681618206 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf04011930 con 0x7faf14105aa0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.490+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf14079280 con 0x7faf14105aa0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.489+0000 7faf117fa700 1 -- 172.21.15.43:0/3681618206 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7faf04003b00 con 0x7faf14105aa0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.490+0000 7faf117fa700 1 -- 172.21.15.43:0/3681618206 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faf0401ad50 con 0x7faf14105aa0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.490+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf14079720 con 0x7faf14105aa0 2024-02-05T07:28:26.193 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.490+0000 7faf117fa700 1 -- 172.21.15.43:0/3681618206 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7faf0402d7d0 con 0x7faf14105aa0 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.490+0000 7faf117fa700 1 -- 172.21.15.43:0/3681618206 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7faf04024730 con 0x7faf14105aa0 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.490+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf14063350 con 0x7faf14105aa0 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.494+0000 7faf117fa700 1 -- 172.21.15.43:0/3681618206 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7faf04019db0 con 0x7faf14105aa0 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.568+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7faf14109ed0 con 0x7faf14105aa0 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.569+0000 7faf117fa700 1 -- 172.21.15.43:0/3681618206 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7faf04035090 con 0x7faf14105aa0 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.570+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 msgr2=0x7faf1407ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.570+0000 7faf1a731700 1 --2- 172.21.15.43:0/3681618206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf1407ab00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7faf04003da0 tx=0x7faf04011e90 comp rx=0 tx=0).stop 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.570+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 shutdown_connections 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.570+0000 7faf1a731700 1 --2- 172.21.15.43:0/3681618206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf14105aa0 0x7faf1407ab00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.570+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 >> 172.21.15.43:0/3681618206 conn(0x7faf14101080 msgr2=0x7faf14101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.570+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 shutdown_connections 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:25.570+0000 7faf1a731700 1 -- 172.21.15.43:0/3681618206 wait complete. 2024-02-05T07:28:26.194 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-02-05T07:28:26.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:26 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:26.303+0000 7fec9125d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:28:26.859 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:26 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:26.590+0000 7fec9125d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:28:26.860 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:26 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:26.857+0000 7fec9125d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:28:27.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:27 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:27.097+0000 7fec9125d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:28:28.298 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:27 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:27.924+0000 7fec9125d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:28:28.549 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:28 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:28.513+0000 7fec9125d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:28:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1535081321' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:28:28.814 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:28 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:28.616+0000 7fec9125d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:28:29.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:28 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:28.846+0000 7fec9125d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:28:29.534 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "f211dd04-c3f7-11ee-95b6-87774f69a715", 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:28:29.535 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:28:29.536 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:28:29.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:28:29.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:28:29.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:28:29.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:28:29.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:29.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:28:29.544 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:29.545 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:28:14.908485+0000", 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:28:29.546 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.650+0000 7f8f99590700 1 Processor -- start 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.650+0000 7f8f99590700 1 -- start start 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.650+0000 7f8f99590700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.650+0000 7f8f99590700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94106490 con 0x7f8f94105aa0 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.650+0000 7f8f92ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.650+0000 7f8f92ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94105ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42426/0 (socket says 172.21.15.43:42426) 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.650+0000 7f8f92ffd700 1 -- 172.21.15.43:0/233686298 learned_addr learned my addr 172.21.15.43:0/233686298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.651+0000 7f8f92ffd700 1 -- 172.21.15.43:0/233686298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f94106ca0 con 0x7f8f94105aa0 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.651+0000 7f8f92ffd700 1 --2- 172.21.15.43:0/233686298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94105ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f840098a0 tx=0x7f8f84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698268323b0568ec server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.651+0000 7f8f91ffb700 1 -- 172.21.15.43:0/233686298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f84004160 con 0x7f8f94105aa0 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.651+0000 7f8f91ffb700 1 -- 172.21.15.43:0/233686298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f8f840042c0 con 0x7f8f94105aa0 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.651+0000 7f8f91ffb700 1 -- 172.21.15.43:0/233686298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f840118a0 con 0x7f8f94105aa0 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 -- 172.21.15.43:0/233686298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 msgr2=0x7f8f94105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 --2- 172.21.15.43:0/233686298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94105ec0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8f840098a0 tx=0x7f8f84009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 -- 172.21.15.43:0/233686298 shutdown_connections 2024-02-05T07:28:29.547 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 --2- 172.21.15.43:0/233686298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94105ec0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 -- 172.21.15.43:0/233686298 >> 172.21.15.43:0/233686298 conn(0x7f8f94101080 msgr2=0x7f8f941034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 -- 172.21.15.43:0/233686298 shutdown_connections 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 -- 172.21.15.43:0/233686298 wait complete. 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 Processor -- start 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.652+0000 7f8f99590700 1 -- start start 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f99590700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f99590700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f94199c80 con 0x7f8f94105aa0 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f92ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f92ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42436/0 (socket says 172.21.15.43:42436) 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f92ffd700 1 -- 172.21.15.43:0/1535081321 learned_addr learned my addr 172.21.15.43:0/1535081321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f92ffd700 1 -- 172.21.15.43:0/1535081321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f84009550 con 0x7f8f94105aa0 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f92ffd700 1 --2- 172.21.15.43:0/1535081321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94199740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f84003f10 tx=0x7f8f84011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f7bfff700 1 -- 172.21.15.43:0/1535081321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f840036a0 con 0x7f8f94105aa0 2024-02-05T07:28:29.548 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f94199e80 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f7bfff700 1 -- 172.21.15.43:0/1535081321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f8f8401a430 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.653+0000 7f8f7bfff700 1 -- 172.21.15.43:0/1535081321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f84024410 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.654+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9419a320 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.654+0000 7f8f7bfff700 1 -- 172.21.15.43:0/1535081321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f8f84024a60 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.654+0000 7f8f7bfff700 1 -- 172.21.15.43:0/1535081321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8f8400b590 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.654+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f94063350 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.657+0000 7f8f7bfff700 1 -- 172.21.15.43:0/1535081321 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8f8401a5a0 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.731+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8f9419ccd0 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.731+0000 7f8f7bfff700 1 -- 172.21.15.43:0/1535081321 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f8f8401a8d0 con 0x7f8f94105aa0 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.732+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 msgr2=0x7f8f94199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.732+0000 7f8f99590700 1 --2- 172.21.15.43:0/1535081321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94199740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8f84003f10 tx=0x7f8f84011940 comp rx=0 tx=0).stop 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.732+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 shutdown_connections 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.732+0000 7f8f99590700 1 --2- 172.21.15.43:0/1535081321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f94105aa0 0x7f8f94199740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:29.549 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.732+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 >> 172.21.15.43:0/1535081321 conn(0x7f8f94101080 msgr2=0x7f8f94101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:29.550 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.733+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 shutdown_connections 2024-02-05T07:28:29.550 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:28.733+0000 7f8f99590700 1 -- 172.21.15.43:0/1535081321 wait complete. 2024-02-05T07:28:29.550 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (3/15)... 2024-02-05T07:28:30.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: Activating manager daemon y 2024-02-05T07:28:30.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: mgrmap e2: y(active, starting, since 0.00319417s) 2024-02-05T07:28:30.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: Manager daemon y is now available 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' 2024-02-05T07:28:30.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:29 smithi043 ceph-mon[29879]: from='mgr.14100 172.21.15.43:0/1223276206' entity='mgr.y' 2024-02-05T07:28:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:30 smithi043 ceph-mon[29879]: mgrmap e3: y(active, since 1.00721s) 2024-02-05T07:28:33.149 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:32 smithi043 ceph-mon[29879]: mgrmap e4: y(active, since 3s) 2024-02-05T07:28:36.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/985351279' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "f211dd04-c3f7-11ee-95b6-87774f69a715", 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-02-05T07:28:38.202 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-05T07:28:38.203 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-05T07:28:38.211 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-05T07:28:38.211 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-05T07:28:38.211 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-05T07:28:38.211 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:38.211 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-05T07:28:38.212 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-02-05T07:28:14.908485+0000", 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.246+0000 7fbee9d63700 1 Processor -- start 2024-02-05T07:28:38.213 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.246+0000 7fbee9d63700 1 -- start start 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.246+0000 7fbee9d63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee4105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.246+0000 7fbee9d63700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4106470 con 0x7fbee4105a80 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.246+0000 7fbee37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee4105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.246+0000 7fbee37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee4105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36372/0 (socket says 172.21.15.43:36372) 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.246+0000 7fbee37fe700 1 -- 172.21.15.43:0/235213640 learned_addr learned my addr 172.21.15.43:0/235213640 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee37fe700 1 -- 172.21.15.43:0/235213640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee4106c80 con 0x7fbee4105a80 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee37fe700 1 --2- 172.21.15.43:0/235213640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee4105ea0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098a0 tx=0x7fbed4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7a703282d10435 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee27fc700 1 -- 172.21.15.43:0/235213640 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed4004160 con 0x7fbee4105a80 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee27fc700 1 -- 172.21.15.43:0/235213640 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fbed40042c0 con 0x7fbee4105a80 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee27fc700 1 -- 172.21.15.43:0/235213640 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed40118a0 con 0x7fbee4105a80 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee9d63700 1 -- 172.21.15.43:0/235213640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 msgr2=0x7fbee4105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee9d63700 1 --2- 172.21.15.43:0/235213640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee4105ea0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbed40098a0 tx=0x7fbed4009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee9d63700 1 -- 172.21.15.43:0/235213640 shutdown_connections 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee9d63700 1 --2- 172.21.15.43:0/235213640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee4105ea0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:38.214 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.247+0000 7fbee9d63700 1 -- 172.21.15.43:0/235213640 >> 172.21.15.43:0/235213640 conn(0x7fbee4101040 msgr2=0x7fbee4103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.248+0000 7fbee9d63700 1 -- 172.21.15.43:0/235213640 shutdown_connections 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.248+0000 7fbee9d63700 1 -- 172.21.15.43:0/235213640 wait complete. 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.248+0000 7fbee9d63700 1 Processor -- start 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.248+0000 7fbee9d63700 1 -- start start 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.248+0000 7fbee9d63700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.248+0000 7fbee9d63700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee4199c30 con 0x7fbee4105a80 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee41996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36376/0 (socket says 172.21.15.43:36376) 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee37fe700 1 -- 172.21.15.43:0/985351279 learned_addr learned my addr 172.21.15.43:0/985351279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee37fe700 1 -- 172.21.15.43:0/985351279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed4009550 con 0x7fbee4105a80 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee37fe700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee41996f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbed4003f10 tx=0x7fbed4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee0ff9700 1 -- 172.21.15.43:0/985351279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed40036a0 con 0x7fbee4105a80 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee0ff9700 1 -- 172.21.15.43:0/985351279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fbed401a430 con 0x7fbee4105a80 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee4199e30 con 0x7fbee4105a80 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee0ff9700 1 -- 172.21.15.43:0/985351279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbed4024410 con 0x7fbee4105a80 2024-02-05T07:28:38.215 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.249+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee419a2d0 con 0x7fbee4105a80 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.250+0000 7fbee0ff9700 1 -- 172.21.15.43:0/985351279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7fbed402d3f0 con 0x7fbee4105a80 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.250+0000 7fbee0ff9700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7fbecc038ee0 0x7fbecc03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.250+0000 7fbee0ff9700 1 -- 172.21.15.43:0/985351279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbed404c8d0 con 0x7fbee4105a80 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.250+0000 7fbee2ffd700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7fbecc038ee0 0x7fbecc03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.250+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee4063350 con 0x7fbee4105a80 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.251+0000 7fbee2ffd700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7fbecc038ee0 0x7fbecc03b3a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006fd0 tx=0x7fbed8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.256+0000 7fbee0ff9700 1 -- 172.21.15.43:0/985351279 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbed4010970 con 0x7fbee4105a80 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.542+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fbee4109e80 con 0x7fbee4105a80 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.542+0000 7fbee0ff9700 1 -- 172.21.15.43:0/985351279 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fbed4010380 con 0x7fbee4105a80 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7fbecc038ee0 msgr2=0x7fbecc03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7fbecc038ee0 0x7fbecc03b3a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbed8006fd0 tx=0x7fbed8006e40 comp rx=0 tx=0).stop 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 msgr2=0x7fbee41996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee41996f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbed4003f10 tx=0x7fbed4011940 comp rx=0 tx=0).stop 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 shutdown_connections 2024-02-05T07:28:38.216 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7fbecc038ee0 0x7fbecc03b3a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 --2- 172.21.15.43:0/985351279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbee4105a80 0x7fbee41996f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.544+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 >> 172.21.15.43:0/985351279 conn(0x7fbee4101040 msgr2=0x7fbee4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.545+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 shutdown_connections 2024-02-05T07:28:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:35.545+0000 7fbee9d63700 1 -- 172.21.15.43:0/985351279 wait complete. 2024-02-05T07:28:38.217 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-02-05T07:28:39.202 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1336482079' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-05T07:28:39.473 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.689+0000 7f13aacce700 1 Processor -- start 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.689+0000 7f13aacce700 1 -- start start 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.689+0000 7f13aacce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a4108960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.689+0000 7f13aacce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40796c0 con 0x7f13a4106570 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.690+0000 7f13a8a6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a4108960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.690+0000 7f13a8a6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a4108960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36384/0 (socket says 172.21.15.43:36384) 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.690+0000 7f13a8a6a700 1 -- 172.21.15.43:0/2720130972 learned_addr learned my addr 172.21.15.43:0/2720130972 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.690+0000 7f13a8a6a700 1 -- 172.21.15.43:0/2720130972 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a4108ea0 con 0x7f13a4106570 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.690+0000 7f13a8a6a700 1 --2- 172.21.15.43:0/2720130972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a4108960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f13980098a0 tx=0x7f1398009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe33577bc0caeb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.690+0000 7f13a37fe700 1 -- 172.21.15.43:0/2720130972 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1398004160 con 0x7f13a4106570 2024-02-05T07:28:39.474 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13a37fe700 1 -- 172.21.15.43:0/2720130972 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f13980042c0 con 0x7f13a4106570 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13a37fe700 1 -- 172.21.15.43:0/2720130972 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13980118a0 con 0x7f13a4106570 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13aacce700 1 -- 172.21.15.43:0/2720130972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 msgr2=0x7f13a4108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13aacce700 1 --2- 172.21.15.43:0/2720130972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a4108960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f13980098a0 tx=0x7f1398009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13aacce700 1 -- 172.21.15.43:0/2720130972 shutdown_connections 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13aacce700 1 --2- 172.21.15.43:0/2720130972 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a4108960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13aacce700 1 -- 172.21.15.43:0/2720130972 >> 172.21.15.43:0/2720130972 conn(0x7f13a40fff80 msgr2=0x7f13a41023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13aacce700 1 -- 172.21.15.43:0/2720130972 shutdown_connections 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.691+0000 7f13aacce700 1 -- 172.21.15.43:0/2720130972 wait complete. 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13aacce700 1 Processor -- start 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13aacce700 1 -- start start 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13aacce700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a41996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13aacce700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a4199c10 con 0x7f13a4106570 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13a8a6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a41996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:39.475 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13a8a6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a41996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36398/0 (socket says 172.21.15.43:36398) 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13a8a6a700 1 -- 172.21.15.43:0/1336482079 learned_addr learned my addr 172.21.15.43:0/1336482079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.692+0000 7f13a8a6a700 1 -- 172.21.15.43:0/1336482079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1398009550 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.693+0000 7f13a8a6a700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a41996d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1398011890 tx=0x7f1398011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.693+0000 7f13a1ffb700 1 -- 172.21.15.43:0/1336482079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13980036a0 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.693+0000 7f13a1ffb700 1 -- 172.21.15.43:0/1336482079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f139801a430 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.693+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a4199e10 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.693+0000 7f13a1ffb700 1 -- 172.21.15.43:0/1336482079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13980193c0 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.693+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a419a2b0 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.694+0000 7f13a1ffb700 1 -- 172.21.15.43:0/1336482079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f1398019640 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.694+0000 7f13a1ffb700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f1394038e90 0x7f139403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.694+0000 7f13a1ffb700 1 -- 172.21.15.43:0/1336482079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f139804cfa0 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.694+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a4109b80 con 0x7f13a4106570 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.694+0000 7f13a3fff700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f1394038e90 0x7f139403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:39.476 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.694+0000 7f13a3fff700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f1394038e90 0x7f139403b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1390006fd0 tx=0x7f1390006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.699+0000 7f13a1ffb700 1 -- 172.21.15.43:0/1336482079 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1398018940 con 0x7f13a4106570 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.901+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f13a4109d40 con 0x7f13a4106570 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.901+0000 7f13a1ffb700 1 -- 172.21.15.43:0/1336482079 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (secure 0 0 0) 0x7f139802d350 con 0x7f13a4106570 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f1394038e90 msgr2=0x7f139403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f1394038e90 0x7f139403b350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1390006fd0 tx=0x7f1390006e40 comp rx=0 tx=0).stop 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 msgr2=0x7f13a41996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a41996d0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1398011890 tx=0x7f1398011970 comp rx=0 tx=0).stop 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 shutdown_connections 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f1394038e90 0x7f139403b350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 --2- 172.21.15.43:0/1336482079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a4106570 0x7f13a41996d0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 >> 172.21.15.43:0/1336482079 conn(0x7f13a40fff80 msgr2=0x7f13a4100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 shutdown_connections 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:38.904+0000 7f13aacce700 1 -- 172.21.15.43:0/1336482079 wait complete. 2024-02-05T07:28:39.477 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-02-05T07:28:41.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2870592140' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-05T07:28:41.175 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:40 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ignoring --setuser ceph since I am not root 2024-02-05T07:28:41.175 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:40 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ignoring --setgroup ceph since I am not root 2024-02-05T07:28:41.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.927+0000 7f0976314700 1 Processor -- start 2024-02-05T07:28:41.424 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.927+0000 7f0976314700 1 -- start start 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.927+0000 7f0976314700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.927+0000 7f0976314700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970106480 con 0x7f0970105a90 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36414/0 (socket says 172.21.15.43:36414) 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096ffff700 1 -- 172.21.15.43:0/805969700 learned_addr learned my addr 172.21.15.43:0/805969700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096ffff700 1 -- 172.21.15.43:0/805969700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0970106c90 con 0x7f0970105a90 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096ffff700 1 --2- 172.21.15.43:0/805969700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970105eb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09580098a0 tx=0x7f0958009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2161163655f91b server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096effd700 1 -- 172.21.15.43:0/805969700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0958004160 con 0x7f0970105a90 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096effd700 1 -- 172.21.15.43:0/805969700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f09580042c0 con 0x7f0970105a90 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.928+0000 7f096effd700 1 -- 172.21.15.43:0/805969700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f09580118a0 con 0x7f0970105a90 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.929+0000 7f0976314700 1 -- 172.21.15.43:0/805969700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 msgr2=0x7f0970105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.929+0000 7f0976314700 1 --2- 172.21.15.43:0/805969700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970105eb0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f09580098a0 tx=0x7f0958009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.929+0000 7f0976314700 1 -- 172.21.15.43:0/805969700 shutdown_connections 2024-02-05T07:28:41.425 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.929+0000 7f0976314700 1 --2- 172.21.15.43:0/805969700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970105eb0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.929+0000 7f0976314700 1 -- 172.21.15.43:0/805969700 >> 172.21.15.43:0/805969700 conn(0x7f0970101090 msgr2=0x7f09701034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.929+0000 7f0976314700 1 -- 172.21.15.43:0/805969700 shutdown_connections 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.929+0000 7f0976314700 1 -- 172.21.15.43:0/805969700 wait complete. 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f0976314700 1 Processor -- start 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f0976314700 1 -- start start 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f0976314700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f0976314700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0970199c70 con 0x7f0970105a90 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f096ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f096ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36422/0 (socket says 172.21.15.43:36422) 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f096ffff700 1 -- 172.21.15.43:0/2870592140 learned_addr learned my addr 172.21.15.43:0/2870592140 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f096ffff700 1 -- 172.21.15.43:0/2870592140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958009550 con 0x7f0970105a90 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.930+0000 7f096ffff700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970199730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0958003da0 tx=0x7f0958011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.931+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0958011930 con 0x7f0970105a90 2024-02-05T07:28:41.426 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.931+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f0958003b00 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.931+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0970199e70 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.931+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095801ad50 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.931+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f097019a310 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.931+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 45921+0+0 (secure 0 0 0) 0x7f095802d440 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.932+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0970063350 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.932+0000 7f096d7fa700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f095c038e90 0x7f095c03b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.932+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f095804cfe0 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.932+0000 7f096f7fe700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f095c038e90 0x7f095c03b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.932+0000 7f096f7fe700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f095c038e90 0x7f095c03b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0960006fd0 tx=0x7f0960006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:39.937+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f095802d6f0 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.199+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f097019cd30 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.867+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f0958018350 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.869+0000 7f096d7fa700 1 -- 172.21.15.43:0/2870592140 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f095804e070 con 0x7f0970105a90 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f095c038e90 msgr2=0x7f095c03b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:41.427 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f095c038e90 0x7f095c03b350 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0960006fd0 tx=0x7f0960006e40 comp rx=0 tx=0).stop 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 msgr2=0x7f0970199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970199730 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0958003da0 tx=0x7f0958011e90 comp rx=0 tx=0).stop 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 shutdown_connections 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f095c038e90 0x7f095c03b350 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 --2- 172.21.15.43:0/2870592140 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0970105a90 0x7f0970199730 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 >> 172.21.15.43:0/2870592140 conn(0x7f0970101090 msgr2=0x7f0970101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 shutdown_connections 2024-02-05T07:28:41.428 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:40.871+0000 7f0976314700 1 -- 172.21.15.43:0/2870592140 wait complete. 2024-02-05T07:28:41.803 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:41 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:41.452+0000 7f13c4adf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:28:41.803 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:41 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:41.548+0000 7f13c4adf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:28:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2870592140' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-05T07:28:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:41 smithi043 ceph-mon[29879]: mgrmap e5: y(active, since 12s) 2024-02-05T07:28:42.461 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:42.221+0000 7f13c4adf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:28:42.461 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:42.316+0000 7f13c4adf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:28:42.461 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:42.459+0000 7f13c4adf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:28:42.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:42.558+0000 7f13c4adf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:28:42.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:42.643+0000 7f13c4adf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:28:43.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2711714935' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-05T07:28:43.210 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:42.910+0000 7f13c4adf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:28:43.210 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:43 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:43.023+0000 7f13c4adf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:28:43.210 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:43 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:43.111+0000 7f13c4adf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:28:43.210 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:43 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:43.208+0000 7f13c4adf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:28:43.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:28:43.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-05T07:28:43.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T07:28:43.315 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b85550700 1 Processor -- start 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b85550700 1 -- start start 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b85550700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b80104db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b85550700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b80105380 con 0x7f7b80104990 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b80104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b80104db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56162/0 (socket says 172.21.15.43:56162) 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b7effd700 1 -- 172.21.15.43:0/3457730324 learned_addr learned my addr 172.21.15.43:0/3457730324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.904+0000 7f7b7effd700 1 -- 172.21.15.43:0/3457730324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b80105b90 con 0x7f7b80104990 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.905+0000 7f7b7effd700 1 --2- 172.21.15.43:0/3457730324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b80104db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7b700098a0 tx=0x7f7b70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d776a83c007cd0a server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.905+0000 7f7b7dffb700 1 -- 172.21.15.43:0/3457730324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b70004160 con 0x7f7b80104990 2024-02-05T07:28:43.316 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.905+0000 7f7b7dffb700 1 -- 172.21.15.43:0/3457730324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7b700042c0 con 0x7f7b80104990 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.905+0000 7f7b7dffb700 1 -- 172.21.15.43:0/3457730324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b700118a0 con 0x7f7b80104990 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.905+0000 7f7b85550700 1 -- 172.21.15.43:0/3457730324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 msgr2=0x7f7b80104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.905+0000 7f7b85550700 1 --2- 172.21.15.43:0/3457730324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b80104db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7b700098a0 tx=0x7f7b70009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.906+0000 7f7b85550700 1 -- 172.21.15.43:0/3457730324 shutdown_connections 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.906+0000 7f7b85550700 1 --2- 172.21.15.43:0/3457730324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b80104db0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.906+0000 7f7b85550700 1 -- 172.21.15.43:0/3457730324 >> 172.21.15.43:0/3457730324 conn(0x7f7b800fffd0 msgr2=0x7f7b801023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.906+0000 7f7b85550700 1 -- 172.21.15.43:0/3457730324 shutdown_connections 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.906+0000 7f7b85550700 1 -- 172.21.15.43:0/3457730324 wait complete. 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.906+0000 7f7b85550700 1 Processor -- start 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.906+0000 7f7b85550700 1 -- start start 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b85550700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b8018f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b85550700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8018fec0 con 0x7f7b80104990 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b8018f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:43.317 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b7effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b8018f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56168/0 (socket says 172.21.15.43:56168) 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b7effd700 1 -- 172.21.15.43:0/2711714935 learned_addr learned my addr 172.21.15.43:0/2711714935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b7effd700 1 -- 172.21.15.43:0/2711714935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b70009550 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b7effd700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b8018f980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b70003da0 tx=0x7f7b70011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b67fff700 1 -- 172.21.15.43:0/2711714935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b70011930 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b67fff700 1 -- 172.21.15.43:0/2711714935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7b70003b00 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b8018e0b0 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b67fff700 1 -- 172.21.15.43:0/2711714935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b7001ad50 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.907+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b8018e550 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.908+0000 7f7b67fff700 1 -- 172.21.15.43:0/2711714935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f7b7002d440 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.909+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b80063350 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.909+0000 7f7b67fff700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 0x7f7b6803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.909+0000 7f7b67fff700 1 -- 172.21.15.43:0/2711714935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7b7004ce40 con 0x7f7b80104990 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.909+0000 7f7b7e7fc700 1 -- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 msgr2=0x7f7b6803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1476147457 2024-02-05T07:28:43.318 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.909+0000 7f7b7e7fc700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 0x7f7b6803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:41.914+0000 7f7b67fff700 1 -- 172.21.15.43:0/2711714935 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b70018940 con 0x7f7b80104990 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.110+0000 7f7b7e7fc700 1 -- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 msgr2=0x7f7b6803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1476147457 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.110+0000 7f7b7e7fc700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 0x7f7b6803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.178+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f7b80193290 con 0x7f7b80104990 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.178+0000 7f7b67fff700 1 -- 172.21.15.43:0/2711714935 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f7b70018350 con 0x7f7b80104990 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 msgr2=0x7f7b6803b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 0x7f7b6803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 msgr2=0x7f7b8018f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b8018f980 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7b70003da0 tx=0x7f7b70011e90 comp rx=0 tx=0).stop 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 shutdown_connections 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f7b68038ee0 0x7f7b6803b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 --2- 172.21.15.43:0/2711714935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7b80104990 0x7f7b8018f980 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.180+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 >> 172.21.15.43:0/2711714935 conn(0x7f7b800fffd0 msgr2=0x7f7b80100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:43.319 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.181+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 shutdown_connections 2024-02-05T07:28:43.320 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:42.181+0000 7f7b85550700 1 -- 172.21.15.43:0/2711714935 wait complete. 2024-02-05T07:28:43.320 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-02-05T07:28:43.320 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-02-05T07:28:43.510 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:43 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:43.388+0000 7f13c4adf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:28:44.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:44 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:44.381+0000 7f13c4adf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:28:44.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:44 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:44.514+0000 7f13c4adf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:28:45.143 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:44 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:44.851+0000 7f13c4adf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:28:45.143 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:44 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:44.943+0000 7f13c4adf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:28:45.143 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:45 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:45.141+0000 7f13c4adf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:28:45.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:45 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:45.329+0000 7f13c4adf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:28:45.881 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:45 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:45.619+0000 7f13c4adf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:28:45.881 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:45 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:45.879+0000 7f13c4adf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:28:46.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:46 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:46.123+0000 7f13c4adf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:28:47.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:46 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:46.952+0000 7f13c4adf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:28:47.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:47.535+0000 7f13c4adf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:28:47.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:47.635+0000 7f13c4adf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:28:48.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:28:47.855+0000 7f13c4adf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:28:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: Active manager daemon y restarted 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: Activating manager daemon y 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: osdmap e2: 0 total, 0 up, 0 in 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: mgrmap e6: y(active, starting, since 0.00305234s) 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: Manager daemon y is now available 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:28:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:28:48.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:47 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:28:49.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: Found migration_current of "None". Setting to last migration. 2024-02-05T07:28:49.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:28:49.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:28:49.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:28:49.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:28:49.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:28:49.188 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:28:49.189 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:48 smithi043 ceph-mon[29879]: mgrmap e7: y(active, since 1.00602s) 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.728+0000 7f49a2806700 1 Processor -- start 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.728+0000 7f49a2806700 1 -- start start 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.728+0000 7f49a2806700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1049a0 0x7f499c104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.728+0000 7f49a2806700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c105390 con 0x7f499c1049a0 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.728+0000 7f499bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1049a0 0x7f499c104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:49.482 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.728+0000 7f499bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1049a0 0x7f499c104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56176/0 (socket says 172.21.15.43:56176) 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.728+0000 7f499bfff700 1 -- 172.21.15.43:0/2290045658 learned_addr learned my addr 172.21.15.43:0/2290045658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.729+0000 7f499bfff700 1 -- 172.21.15.43:0/2290045658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f499c105ba0 con 0x7f499c1049a0 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.729+0000 7f499bfff700 1 --2- 172.21.15.43:0/2290045658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1049a0 0x7f499c104dc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49900098a0 tx=0x7f4990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948e6d42531700a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.729+0000 7f499affd700 1 -- 172.21.15.43:0/2290045658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4990004160 con 0x7f499c1049a0 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.729+0000 7f499affd700 1 -- 172.21.15.43:0/2290045658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f49900042c0 con 0x7f499c1049a0 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.729+0000 7f499affd700 1 -- 172.21.15.43:0/2290045658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49900118a0 con 0x7f499c1049a0 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.730+0000 7f49a2806700 1 -- 172.21.15.43:0/2290045658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1049a0 msgr2=0x7f499c104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.730+0000 7f49a2806700 1 --2- 172.21.15.43:0/2290045658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1049a0 0x7f499c104dc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49900098a0 tx=0x7f4990009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.730+0000 7f49a2806700 1 -- 172.21.15.43:0/2290045658 shutdown_connections 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.730+0000 7f49a2806700 1 --2- 172.21.15.43:0/2290045658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1049a0 0x7f499c104dc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.730+0000 7f49a2806700 1 -- 172.21.15.43:0/2290045658 >> 172.21.15.43:0/2290045658 conn(0x7f499c0fffa0 msgr2=0x7f499c1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.733+0000 7f49a2806700 1 -- 172.21.15.43:0/2290045658 shutdown_connections 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.733+0000 7f49a2806700 1 -- 172.21.15.43:0/2290045658 wait complete. 2024-02-05T07:28:49.483 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.734+0000 7f49a2806700 1 Processor -- start 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.734+0000 7f49a2806700 1 -- start start 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.735+0000 7f49a2806700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c198880 0x7f499c198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.735+0000 7f49a2806700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1991e0 con 0x7f499c198880 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.741+0000 7f499bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c198880 0x7f499c198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.741+0000 7f499bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c198880 0x7f499c198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56190/0 (socket says 172.21.15.43:56190) 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.741+0000 7f499bfff700 1 -- 172.21.15.43:0/3695379001 learned_addr learned my addr 172.21.15.43:0/3695379001 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.748+0000 7f499bfff700 1 -- 172.21.15.43:0/3695379001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990009550 con 0x7f499c198880 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.755+0000 7f499bfff700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c198880 0x7f499c198ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4990006af0 tx=0x7f4990011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.758+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49900036a0 con 0x7f499c198880 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.758+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f499001a460 con 0x7f499c198880 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.758+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f499001b440 con 0x7f499c198880 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.759+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c1993e0 con 0x7f499c198880 2024-02-05T07:28:49.484 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.759+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c19bf50 con 0x7f499c198880 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.772+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f4990021070 con 0x7f499c198880 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.772+0000 7f49997fa700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.772+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 --> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f498403bb00 con 0x7f4984038f30 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.772+0000 7f499b7fe700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 msgr2=0x7f498403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1476147457 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.772+0000 7f499b7fe700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.775+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4990053060 con 0x7f499c198880 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.973+0000 7f499b7fe700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 msgr2=0x7f498403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1476147457 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:43.973+0000 7f499b7fe700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:44.374+0000 7f499b7fe700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 msgr2=0x7f498403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1476147457 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:44.374+0000 7f499b7fe700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:45.175+0000 7f499b7fe700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 msgr2=0x7f498403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1476147457 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:45.175+0000 7f499b7fe700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:46.776+0000 7f499b7fe700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 msgr2=0x7f498403b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/1476147457 2024-02-05T07:28:49.485 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:46.776+0000 7f499b7fe700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:47.871+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 6) v1 ==== 45699+0+0 (secure 0 0 0) 0x7f499001a920 con 0x7f499c198880 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:47.871+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 msgr2=0x7f498403b3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:47.871+0000 7f49997fa700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.872+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f499000ecf0 con 0x7f499c198880 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.872+0000 7f49997fa700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f498403c810 0x7f498403ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.873+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f498403bb00 con 0x7f498403c810 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.873+0000 7f499b7fe700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f498403c810 0x7f498403ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.874+0000 7f499b7fe700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f498403c810 0x7f498403ec00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f498c003a10 tx=0x7f498c009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.875+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f498403bb00 con 0x7f498403c810 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.882+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f499c063350 con 0x7f498403c810 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.882+0000 7f49997fa700 1 -- 172.21.15.43:0/3695379001 <== mgr.14120 v2:172.21.15.43:6800/4155203878 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f499c063350 con 0x7f498403c810 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.882+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f498403c810 msgr2=0x7f498403ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.882+0000 7f49a2806700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f498403c810 0x7f498403ec00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f498c003a10 tx=0x7f498c009290 comp rx=0 tx=0).stop 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.882+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c198880 msgr2=0x7f499c198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:49.486 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.882+0000 7f49a2806700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c198880 0x7f499c198ca0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4990006af0 tx=0x7f4990011940 comp rx=0 tx=0).stop 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.883+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 shutdown_connections 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.883+0000 7f49a2806700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f498403c810 0x7f498403ec00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.883+0000 7f49a2806700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c198880 0x7f499c198ca0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.883+0000 7f49a2806700 1 --2- 172.21.15.43:0/3695379001 >> [v2:172.21.15.43:6800/1476147457,v1:172.21.15.43:6801/1476147457] conn(0x7f4984038f30 0x7f498403b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.883+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 >> 172.21.15.43:0/3695379001 conn(0x7f499c0fffa0 msgr2=0x7f499c100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.883+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 shutdown_connections 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:48.883+0000 7f49a2806700 1 -- 172.21.15.43:0/3695379001 wait complete. 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-02-05T07:28:49.487 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-02-05T07:28:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:49 smithi043 ceph-mon[29879]: [05/Feb/2024:07:28:48] ENGINE Bus STARTING 2024-02-05T07:28:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:49 smithi043 ceph-mon[29879]: [05/Feb/2024:07:28:48] ENGINE Serving on http://172.21.15.43:8765 2024-02-05T07:28:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:49 smithi043 ceph-mon[29879]: [05/Feb/2024:07:28:48] ENGINE Serving on https://172.21.15.43:7150 2024-02-05T07:28:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:49 smithi043 ceph-mon[29879]: [05/Feb/2024:07:28:48] ENGINE Bus STARTED 2024-02-05T07:28:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:49 smithi043 ceph-mon[29879]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-05T07:28:50.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:49 smithi043 ceph-mon[29879]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-05T07:28:50.728 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.923+0000 7f7169f1a700 1 Processor -- start 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.923+0000 7f7169f1a700 1 -- start start 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.923+0000 7f7169f1a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f7164108970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.923+0000 7f7169f1a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164078e10 con 0x7f7164106580 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.923+0000 7f71637fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f7164108970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.923+0000 7f71637fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f7164108970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56264/0 (socket says 172.21.15.43:56264) 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.923+0000 7f71637fe700 1 -- 172.21.15.43:0/2029532630 learned_addr learned my addr 172.21.15.43:0/2029532630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.924+0000 7f71637fe700 1 -- 172.21.15.43:0/2029532630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7164108eb0 con 0x7f7164106580 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.924+0000 7f71637fe700 1 --2- 172.21.15.43:0/2029532630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f7164108970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7154009e80 tx=0x7f715400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1f4d048f764262 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.924+0000 7f71627fc700 1 -- 172.21.15.43:0/2029532630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f715400e070 con 0x7f7164106580 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.924+0000 7f71627fc700 1 -- 172.21.15.43:0/2029532630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7154003e80 con 0x7f7164106580 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.924+0000 7f71627fc700 1 -- 172.21.15.43:0/2029532630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7154004370 con 0x7f7164106580 2024-02-05T07:28:50.729 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.924+0000 7f7169f1a700 1 -- 172.21.15.43:0/2029532630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 msgr2=0x7f7164108970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.924+0000 7f7169f1a700 1 --2- 172.21.15.43:0/2029532630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f7164108970 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7154009e80 tx=0x7f715400b330 comp rx=0 tx=0).stop 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 -- 172.21.15.43:0/2029532630 shutdown_connections 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 --2- 172.21.15.43:0/2029532630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f7164108970 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 -- 172.21.15.43:0/2029532630 >> 172.21.15.43:0/2029532630 conn(0x7f71640fffd0 msgr2=0x7f71641023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 -- 172.21.15.43:0/2029532630 shutdown_connections 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 -- 172.21.15.43:0/2029532630 wait complete. 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 Processor -- start 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 -- start start 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f71641997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.925+0000 7f7169f1a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7164199d20 con 0x7f7164106580 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f71637fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f71641997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f71637fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f71641997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56274/0 (socket says 172.21.15.43:56274) 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f71637fe700 1 -- 172.21.15.43:0/2711952370 learned_addr learned my addr 172.21.15.43:0/2711952370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f71637fe700 1 -- 172.21.15.43:0/2711952370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7154009b30 con 0x7f7164106580 2024-02-05T07:28:50.730 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f71637fe700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f71641997e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7154003c50 tx=0x7f7154003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f7160ff9700 1 -- 172.21.15.43:0/2711952370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f715400e040 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7164199f20 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f7160ff9700 1 -- 172.21.15.43:0/2711952370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f71540094d0 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f7160ff9700 1 -- 172.21.15.43:0/2711952370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71540127c0 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.926+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716419a420 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.927+0000 7f7160ff9700 1 -- 172.21.15.43:0/2711952370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 7) v1 ==== 45826+0+0 (secure 0 0 0) 0x7f715401c650 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.927+0000 7f7160ff9700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f7150038dd0 0x7f715003b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.927+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7164109b40 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.927+0000 7f7160ff9700 1 -- 172.21.15.43:0/2711952370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f715404d000 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.927+0000 7f7162ffd700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f7150038dd0 0x7f715003b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.928+0000 7f7162ffd700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f7150038dd0 0x7f715003b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f714c006fd0 tx=0x7f714c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:49.933+0000 7f7160ff9700 1 -- 172.21.15.43:0/2711952370 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f715401cc40 con 0x7f7164106580 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.151+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f71641011d0 con 0x7f7150038dd0 2024-02-05T07:28:50.731 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.156+0000 7f7160ff9700 1 -- 172.21.15.43:0/2711952370 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f71641011d0 con 0x7f7150038dd0 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f7150038dd0 msgr2=0x7f715003b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f7150038dd0 0x7f715003b290 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f714c006fd0 tx=0x7f714c006e40 comp rx=0 tx=0).stop 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 msgr2=0x7f71641997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f71641997e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7154003c50 tx=0x7f7154003d30 comp rx=0 tx=0).stop 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 shutdown_connections 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f7150038dd0 0x7f715003b290 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 --2- 172.21.15.43:0/2711952370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7164106580 0x7f71641997e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.158+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 >> 172.21.15.43:0/2711952370 conn(0x7f71640fffd0 msgr2=0x7f7164100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.159+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 shutdown_connections 2024-02-05T07:28:50.732 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:50.159+0000 7f7169f1a700 1 -- 172.21.15.43:0/2711952370 wait complete. 2024-02-05T07:28:51.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:51 smithi043 ceph-mon[29879]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:28:51.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:51 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:28:51.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:51 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:28:51.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:51 smithi043 ceph-mon[29879]: mgrmap e8: y(active, since 2s) 2024-02-05T07:28:52.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-05T07:28:52.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.210+0000 7fed30251700 1 Processor -- start 2024-02-05T07:28:52.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.210+0000 7fed30251700 1 -- start start 2024-02-05T07:28:52.009 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.211+0000 7fed30251700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28106360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.211+0000 7fed30251700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed28106930 con 0x7fed28105f40 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.211+0000 7fed2dfed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.211+0000 7fed2dfed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28106360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56282/0 (socket says 172.21.15.43:56282) 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.211+0000 7fed2dfed700 1 -- 172.21.15.43:0/700332507 learned_addr learned my addr 172.21.15.43:0/700332507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.211+0000 7fed2dfed700 1 -- 172.21.15.43:0/700332507 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28104ba0 con 0x7fed28105f40 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.211+0000 7fed2dfed700 1 --2- 172.21.15.43:0/700332507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28106360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed180098a0 tx=0x7fed18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d024c11a7973c727 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed2cfeb700 1 -- 172.21.15.43:0/700332507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed18004160 con 0x7fed28105f40 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed2cfeb700 1 -- 172.21.15.43:0/700332507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fed180042c0 con 0x7fed28105f40 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed2cfeb700 1 -- 172.21.15.43:0/700332507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed180118a0 con 0x7fed28105f40 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed30251700 1 -- 172.21.15.43:0/700332507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 msgr2=0x7fed28106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed30251700 1 --2- 172.21.15.43:0/700332507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28106360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fed180098a0 tx=0x7fed18009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed30251700 1 -- 172.21.15.43:0/700332507 shutdown_connections 2024-02-05T07:28:52.010 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed30251700 1 --2- 172.21.15.43:0/700332507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28106360 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed30251700 1 -- 172.21.15.43:0/700332507 >> 172.21.15.43:0/700332507 conn(0x7fed280ff750 msgr2=0x7fed28101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed30251700 1 -- 172.21.15.43:0/700332507 shutdown_connections 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.212+0000 7fed30251700 1 -- 172.21.15.43:0/700332507 wait complete. 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed30251700 1 Processor -- start 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed30251700 1 -- start start 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed30251700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed30251700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed28199c80 con 0x7fed28105f40 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed2dfed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed2dfed700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56290/0 (socket says 172.21.15.43:56290) 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed2dfed700 1 -- 172.21.15.43:0/3203103777 learned_addr learned my addr 172.21.15.43:0/3203103777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.213+0000 7fed2dfed700 1 -- 172.21.15.43:0/3203103777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed18009550 con 0x7fed28105f40 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.214+0000 7fed2dfed700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28199740 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed18003da0 tx=0x7fed18011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.214+0000 7fed1effd700 1 -- 172.21.15.43:0/3203103777 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed18011930 con 0x7fed28105f40 2024-02-05T07:28:52.011 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.214+0000 7fed1effd700 1 -- 172.21.15.43:0/3203103777 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fed18003b00 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.214+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed28199e80 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.214+0000 7fed1effd700 1 -- 172.21.15.43:0/3203103777 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed1801ad50 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.214+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2819a320 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.215+0000 7fed1effd700 1 -- 172.21.15.43:0/3203103777 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fed180036a0 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.215+0000 7fed1effd700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fed14038ee0 0x7fed1403b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.215+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed281938d0 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.215+0000 7fed2d7ec700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fed14038ee0 0x7fed1403b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.215+0000 7fed1effd700 1 -- 172.21.15.43:0/3203103777 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fed1804ce70 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.215+0000 7fed2d7ec700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fed14038ee0 0x7fed1403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed24006fd0 tx=0x7fed24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.220+0000 7fed1effd700 1 -- 172.21.15.43:0/3203103777 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed1801a630 con 0x7fed28105f40 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.422+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fed2802d7e0 con 0x7fed14038ee0 2024-02-05T07:28:52.012 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.423+0000 7fed1effd700 1 -- 172.21.15.43:0/3203103777 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fed2802d7e0 con 0x7fed14038ee0 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fed14038ee0 msgr2=0x7fed1403b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fed14038ee0 0x7fed1403b3a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed24006fd0 tx=0x7fed24006e40 comp rx=0 tx=0).stop 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 msgr2=0x7fed28199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28199740 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fed18003da0 tx=0x7fed18011e90 comp rx=0 tx=0).stop 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 shutdown_connections 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fed14038ee0 0x7fed1403b3a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 --2- 172.21.15.43:0/3203103777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed28105f40 0x7fed28199740 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 >> 172.21.15.43:0/3203103777 conn(0x7fed280ff750 msgr2=0x7fed281005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 shutdown_connections 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:51.425+0000 7fed30251700 1 -- 172.21.15.43:0/3203103777 wait complete. 2024-02-05T07:28:52.013 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-02-05T07:28:52.259 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:52 smithi043 ceph-mon[29879]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:28:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-mon[29879]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:28:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-mon[29879]: Generating ssh key... 2024-02-05T07:28:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:28:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: Generating public/private rsa key pair. 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: Your identification has been saved in /tmp/tmpeugyd1wm/key. 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: Your public key has been saved in /tmp/tmpeugyd1wm/key.pub. 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: The key fingerprint is: 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: SHA256:mHzbDUmcjrltUfrTzictpo4gOr03xznP7qS0zTwKwfA ceph-f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: The key's randomart image is: 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: +---[RSA 3072]----+ 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | | 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | . . | 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | . + . | 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | . * = + | 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | + E * | 2024-02-05T07:28:53.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | . * = . | 2024-02-05T07:28:53.315 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | .. =.=.= .. | 2024-02-05T07:28:53.315 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | ....o==X.++ o| 2024-02-05T07:28:53.315 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: | ...o o=B@+o+ | 2024-02-05T07:28:53.315 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:28:53 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: +----[SHA256]-----+ 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.484+0000 7fbcebebf700 1 Processor -- start 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.484+0000 7fbcebebf700 1 -- start start 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbcebebf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce4079b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbcebebf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce407a120 con 0x7fbce407b6f0 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce9c5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce4079b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce9c5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce4079b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37034/0 (socket says 172.21.15.43:37034) 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce9c5b700 1 -- 172.21.15.43:0/24337726 learned_addr learned my addr 172.21.15.43:0/24337726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce9c5b700 1 -- 172.21.15.43:0/24337726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce407a260 con 0x7fbce407b6f0 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce9c5b700 1 --2- 172.21.15.43:0/24337726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce4079b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be153927ce6d028 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce8c59700 1 -- 172.21.15.43:0/24337726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd8004160 con 0x7fbce407b6f0 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce8c59700 1 -- 172.21.15.43:0/24337726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fbcd80042c0 con 0x7fbce407b6f0 2024-02-05T07:28:53.640 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.485+0000 7fbce8c59700 1 -- 172.21.15.43:0/24337726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80118a0 con 0x7fbce407b6f0 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.486+0000 7fbcebebf700 1 -- 172.21.15.43:0/24337726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 msgr2=0x7fbce4079b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.486+0000 7fbcebebf700 1 --2- 172.21.15.43:0/24337726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce4079b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbcd80098a0 tx=0x7fbcd8009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.486+0000 7fbcebebf700 1 -- 172.21.15.43:0/24337726 shutdown_connections 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.486+0000 7fbcebebf700 1 --2- 172.21.15.43:0/24337726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce4079b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.486+0000 7fbcebebf700 1 -- 172.21.15.43:0/24337726 >> 172.21.15.43:0/24337726 conn(0x7fbce4101040 msgr2=0x7fbce4103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.486+0000 7fbcebebf700 1 -- 172.21.15.43:0/24337726 shutdown_connections 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.486+0000 7fbcebebf700 1 -- 172.21.15.43:0/24337726 wait complete. 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbcebebf700 1 Processor -- start 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbcebebf700 1 -- start start 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbcebebf700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbcebebf700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce419be30 con 0x7fbce407b6f0 2024-02-05T07:28:53.641 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbce9c5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbce9c5b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37046/0 (socket says 172.21.15.43:37046) 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbce9c5b700 1 -- 172.21.15.43:0/1111747630 learned_addr learned my addr 172.21.15.43:0/1111747630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbce9c5b700 1 -- 172.21.15.43:0/1111747630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd8009550 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbce9c5b700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce419b8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8011890 tx=0x7fbcd8011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.487+0000 7fbcd6ffd700 1 -- 172.21.15.43:0/1111747630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80036a0 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.488+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce419c030 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.488+0000 7fbcd6ffd700 1 -- 172.21.15.43:0/1111747630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fbcd801a430 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.488+0000 7fbcd6ffd700 1 -- 172.21.15.43:0/1111747630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbcd80193c0 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.488+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce419c4d0 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.488+0000 7fbcd6ffd700 1 -- 172.21.15.43:0/1111747630 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fbcd801a5a0 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.488+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce410bdb0 con 0x7fbce407b6f0 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.489+0000 7fbcd6ffd700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fbcd0038ee0 0x7fbcd003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:53.642 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.489+0000 7fbcd6ffd700 1 -- 172.21.15.43:0/1111747630 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fbcd804cfa0 con 0x7fbce407b6f0 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.489+0000 7fbce945a700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fbcd0038ee0 0x7fbcd003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.489+0000 7fbce945a700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fbcd0038ee0 0x7fbcd003b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006fd0 tx=0x7fbce0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.494+0000 7fbcd6ffd700 1 -- 172.21.15.43:0/1111747630 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcd8080d20 con 0x7fbce407b6f0 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:52.696+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fbce4102280 con 0x7fbcd0038ee0 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.033+0000 7fbcd6ffd700 1 -- 172.21.15.43:0/1111747630 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fbce4102280 con 0x7fbcd0038ee0 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fbcd0038ee0 msgr2=0x7fbcd003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fbcd0038ee0 0x7fbcd003b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006fd0 tx=0x7fbce0006e40 comp rx=0 tx=0).stop 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 msgr2=0x7fbce419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce419b8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8011890 tx=0x7fbcd8011970 comp rx=0 tx=0).stop 2024-02-05T07:28:53.643 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 shutdown_connections 2024-02-05T07:28:53.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fbcd0038ee0 0x7fbcd003b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:53.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 --2- 172.21.15.43:0/1111747630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce407b6f0 0x7fbce419b8f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:53.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 >> 172.21.15.43:0/1111747630 conn(0x7fbce4101040 msgr2=0x7fbce4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:53.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 shutdown_connections 2024-02-05T07:28:53.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:53.035+0000 7fbcebebf700 1 -- 172.21.15.43:0/1111747630 wait complete. 2024-02-05T07:28:54.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXNT6KCVmV9DpONF9l0yPtMZE/PQFKKzd1WVvhGyrQq1F4kPWTOmxXBPBEfGqRcqImxXNjzi7aMj7PiPW5fyKxAH32zD+Ac/VqGgzDHKOJA+5Quawfcl1nLU4d2hB8wEUzN1Sc1spNadYszKQxcIrcqXh5hmg0aGXZ6dKBgx+B9CrNj6BlVOaOnn4wmgjOi5jIrmUnplCR+edBFSzCfYXD22GFAwkZjvf/3Y7J4Sj53ZJwlnjlfxZriH+lhoLGf8grTD4Y3Fy91CszWOYnE+Su4Hv+FUJYQxzwnFESSrqkWnTyNa6+c5h4h3yEee2PtRFvYBl+zMjIGpgOontx8PxTx8NtouSqkHy2Uzf2prXcc4Q3MDTbCVEQi2aEek3IMEaR8ktzetIopIwAgKC2cFgkrVTaWKYmTdfx22/HY1QdHAss/zffFMTAH/lYqmTSjq66ibralBUr50XLJJ+6tqvNp07xUfk0ZC0C0j7fzQDQzcGQ2hGlj2Ub3Tx7Bs0MdWc= ceph-f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:28:54.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43ecb8b700 1 Processor -- start 2024-02-05T07:28:54.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43ecb8b700 1 -- start start 2024-02-05T07:28:54.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43ecb8b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e8105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43ecb8b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8106470 con 0x7f43e8105a80 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43e659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e8105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43e659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e8105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37058/0 (socket says 172.21.15.43:37058) 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43e659c700 1 -- 172.21.15.43:0/796363504 learned_addr learned my addr 172.21.15.43:0/796363504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.098+0000 7f43e659c700 1 -- 172.21.15.43:0/796363504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e8106c80 con 0x7f43e8105a80 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.099+0000 7f43e659c700 1 --2- 172.21.15.43:0/796363504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e8105ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43d80098a0 tx=0x7f43d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bdc06ee452781ad server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.099+0000 7f43e559a700 1 -- 172.21.15.43:0/796363504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43d8004160 con 0x7f43e8105a80 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.099+0000 7f43e559a700 1 -- 172.21.15.43:0/796363504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f43d80042c0 con 0x7f43e8105a80 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.099+0000 7f43e559a700 1 -- 172.21.15.43:0/796363504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43d80118a0 con 0x7f43e8105a80 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.099+0000 7f43ecb8b700 1 -- 172.21.15.43:0/796363504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 msgr2=0x7f43e8105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:54.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.099+0000 7f43ecb8b700 1 --2- 172.21.15.43:0/796363504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e8105ea0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f43d80098a0 tx=0x7f43d8009bb0 comp rx=0 tx=0).stop 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 -- 172.21.15.43:0/796363504 shutdown_connections 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 --2- 172.21.15.43:0/796363504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e8105ea0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 -- 172.21.15.43:0/796363504 >> 172.21.15.43:0/796363504 conn(0x7f43e8101040 msgr2=0x7f43e8103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 -- 172.21.15.43:0/796363504 shutdown_connections 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 -- 172.21.15.43:0/796363504 wait complete. 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 Processor -- start 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 -- start start 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e807b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.100+0000 7f43ecb8b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e807b920 con 0x7f43e8105a80 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43e659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e807b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43e659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e807b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37074/0 (socket says 172.21.15.43:37074) 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43e659c700 1 -- 172.21.15.43:0/1007573002 learned_addr learned my addr 172.21.15.43:0/1007573002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43e659c700 1 -- 172.21.15.43:0/1007573002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8009550 con 0x7f43e8105a80 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43e659c700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e807b3e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f43d8003da0 tx=0x7f43d8011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:54.849 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43d77fe700 1 -- 172.21.15.43:0/1007573002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43d8011930 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43d77fe700 1 -- 172.21.15.43:0/1007573002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f43d8003b00 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e8079b10 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43d77fe700 1 -- 172.21.15.43:0/1007573002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f43d801ad50 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.101+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e8079fb0 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.102+0000 7f43d77fe700 1 -- 172.21.15.43:0/1007573002 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f43d802d440 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.102+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e8063350 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.102+0000 7f43d77fe700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f43d0038ee0 0x7f43d003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.102+0000 7f43d77fe700 1 -- 172.21.15.43:0/1007573002 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f43d804ce10 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.102+0000 7f43e5d9b700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f43d0038ee0 0x7f43d003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.103+0000 7f43e5d9b700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f43d0038ee0 0x7f43d003b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43dc006fd0 tx=0x7f43dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.107+0000 7f43d77fe700 1 -- 172.21.15.43:0/1007573002 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43d8018710 con 0x7f43e8105a80 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.309+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f43e8109e80 con 0x7f43d0038ee0 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.310+0000 7f43d77fe700 1 -- 172.21.15.43:0/1007573002 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f43e8109e80 con 0x7f43d0038ee0 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f43d0038ee0 msgr2=0x7f43d003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:54.850 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f43d0038ee0 0x7f43d003b3a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f43dc006fd0 tx=0x7f43dc006e40 comp rx=0 tx=0).stop 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 msgr2=0x7f43e807b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e807b3e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f43d8003da0 tx=0x7f43d8011e90 comp rx=0 tx=0).stop 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 shutdown_connections 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f43d0038ee0 0x7f43d003b3a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 --2- 172.21.15.43:0/1007573002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e8105a80 0x7f43e807b3e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 >> 172.21.15.43:0/1007573002 conn(0x7f43e8101040 msgr2=0x7f43e8101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 shutdown_connections 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:54.312+0000 7f43ecb8b700 1 -- 172.21.15.43:0/1007573002 wait complete. 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-02-05T07:28:54.851 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-02-05T07:28:56.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:55 smithi043 ceph-mon[29879]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:28:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:56 smithi043 ceph-mon[29879]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:28:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:28:57 smithi043 ceph-mon[29879]: Deploying cephadm binary to smithi043 2024-02-05T07:29:00.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-02-05T07:29:00.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbd06cc700 1 Processor -- start 2024-02-05T07:29:00.529 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbd06cc700 1 -- start start 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbd06cc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbd06cc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc8106470 con 0x7fcbc8105a80 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbce468700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbce468700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37090/0 (socket says 172.21.15.43:37090) 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbce468700 1 -- 172.21.15.43:0/1033518296 learned_addr learned my addr 172.21.15.43:0/1033518296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.287+0000 7fcbce468700 1 -- 172.21.15.43:0/1033518296 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc8106c80 con 0x7fcbc8105a80 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbce468700 1 --2- 172.21.15.43:0/1033518296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8105ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40098a0 tx=0x7fcbc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4970c521f2c9fb server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbcd466700 1 -- 172.21.15.43:0/1033518296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc4004160 con 0x7fcbc8105a80 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbcd466700 1 -- 172.21.15.43:0/1033518296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fcbc40042c0 con 0x7fcbc8105a80 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbcd466700 1 -- 172.21.15.43:0/1033518296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc40118a0 con 0x7fcbc8105a80 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbd06cc700 1 -- 172.21.15.43:0/1033518296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 msgr2=0x7fcbc8105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbd06cc700 1 --2- 172.21.15.43:0/1033518296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8105ea0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcbc40098a0 tx=0x7fcbc4009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbd06cc700 1 -- 172.21.15.43:0/1033518296 shutdown_connections 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbd06cc700 1 --2- 172.21.15.43:0/1033518296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8105ea0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:00.530 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbd06cc700 1 -- 172.21.15.43:0/1033518296 >> 172.21.15.43:0/1033518296 conn(0x7fcbc8101040 msgr2=0x7fcbc8103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbd06cc700 1 -- 172.21.15.43:0/1033518296 shutdown_connections 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.288+0000 7fcbd06cc700 1 -- 172.21.15.43:0/1033518296 wait complete. 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.289+0000 7fcbd06cc700 1 Processor -- start 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.289+0000 7fcbd06cc700 1 -- start start 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.289+0000 7fcbd06cc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.289+0000 7fcbd06cc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc8199c60 con 0x7fcbc8105a80 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.289+0000 7fcbce468700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.289+0000 7fcbce468700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37104/0 (socket says 172.21.15.43:37104) 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.289+0000 7fcbce468700 1 -- 172.21.15.43:0/3719190815 learned_addr learned my addr 172.21.15.43:0/3719190815 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.290+0000 7fcbce468700 1 -- 172.21.15.43:0/3719190815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbc4009550 con 0x7fcbc8105a80 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.290+0000 7fcbce468700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8199720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4003f10 tx=0x7fcbc4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.290+0000 7fcbbf7fe700 1 -- 172.21.15.43:0/3719190815 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc40036a0 con 0x7fcbc8105a80 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.290+0000 7fcbbf7fe700 1 -- 172.21.15.43:0/3719190815 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fcbc401a430 con 0x7fcbc8105a80 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.290+0000 7fcbbf7fe700 1 -- 172.21.15.43:0/3719190815 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcbc4024410 con 0x7fcbc8105a80 2024-02-05T07:29:00.531 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.290+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc8199e60 con 0x7fcbc8105a80 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.290+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc819a300 con 0x7fcbc8105a80 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.291+0000 7fcbbf7fe700 1 -- 172.21.15.43:0/3719190815 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7fcbc401a5a0 con 0x7fcbc8105a80 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.291+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc8063350 con 0x7fcbc8105a80 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.291+0000 7fcbbf7fe700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fcbb4038f30 0x7fcbb403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.291+0000 7fcbbf7fe700 1 -- 172.21.15.43:0/3719190815 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcbc404ccb0 con 0x7fcbc8105a80 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.291+0000 7fcbcdc67700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fcbb4038f30 0x7fcbb403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.292+0000 7fcbcdc67700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fcbb4038f30 0x7fcbb403b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8006fd0 tx=0x7fcbb8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.296+0000 7fcbbf7fe700 1 -- 172.21.15.43:0/3719190815 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcbc4018530 con 0x7fcbc8105a80 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:55.504+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbc8102240 con 0x7fcbb4038f30 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.904+0000 7fcbbf7fe700 1 -- 172.21.15.43:0/3719190815 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fcbc8102240 con 0x7fcbb4038f30 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fcbb4038f30 msgr2=0x7fcbb403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fcbb4038f30 0x7fcbb403b3f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcbb8006fd0 tx=0x7fcbb8006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 msgr2=0x7fcbc8199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:00.532 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8199720 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcbc4003f10 tx=0x7fcbc4011940 comp rx=0 tx=0).stop 2024-02-05T07:29:00.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 shutdown_connections 2024-02-05T07:29:00.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fcbb4038f30 0x7fcbb403b3f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:00.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 --2- 172.21.15.43:0/3719190815 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbc8105a80 0x7fcbc8199720 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:00.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 >> 172.21.15.43:0/3719190815 conn(0x7fcbc8101040 msgr2=0x7fcbc8101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:00.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.906+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 shutdown_connections 2024-02-05T07:29:00.533 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:28:59.907+0000 7fcbd06cc700 1 -- 172.21.15.43:0/3719190815 wait complete. 2024-02-05T07:29:00.533 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-02-05T07:29:01.242 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:00 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:29:01.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:00 smithi043 ceph-mon[29879]: Added host smithi043 2024-02-05T07:29:01.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:00 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-05T07:29:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.011+0000 7f9d254b3700 1 Processor -- start 2024-02-05T07:29:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.011+0000 7f9d254b3700 1 -- start start 2024-02-05T07:29:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.011+0000 7f9d254b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d20108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:02.266 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.011+0000 7f9d254b3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d200796e0 con 0x7f9d20106550 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d20108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d20108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37116/0 (socket says 172.21.15.43:37116) 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1effd700 1 -- 172.21.15.43:0/3663866451 learned_addr learned my addr 172.21.15.43:0/3663866451 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1effd700 1 -- 172.21.15.43:0/3663866451 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d20108e80 con 0x7f9d20106550 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1effd700 1 --2- 172.21.15.43:0/3663866451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d20108940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009e80 tx=0x7f9d1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1f904225f21c4d server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1dffb700 1 -- 172.21.15.43:0/3663866451 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d1000e070 con 0x7f9d20106550 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1dffb700 1 -- 172.21.15.43:0/3663866451 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f9d10003e80 con 0x7f9d20106550 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.012+0000 7f9d1dffb700 1 -- 172.21.15.43:0/3663866451 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d10004370 con 0x7f9d20106550 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 -- 172.21.15.43:0/3663866451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 msgr2=0x7f9d20108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 --2- 172.21.15.43:0/3663866451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d20108940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009e80 tx=0x7f9d1000b330 comp rx=0 tx=0).stop 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 -- 172.21.15.43:0/3663866451 shutdown_connections 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 --2- 172.21.15.43:0/3663866451 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d20108940 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 -- 172.21.15.43:0/3663866451 >> 172.21.15.43:0/3663866451 conn(0x7f9d200fff80 msgr2=0x7f9d201023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:02.267 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 -- 172.21.15.43:0/3663866451 shutdown_connections 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 -- 172.21.15.43:0/3663866451 wait complete. 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 Processor -- start 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.013+0000 7f9d254b3700 1 -- start start 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d254b3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d2019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d254b3700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2019bfa0 con 0x7f9d20106550 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d2019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d1effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d2019ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37124/0 (socket says 172.21.15.43:37124) 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d1effd700 1 -- 172.21.15.43:0/258050342 learned_addr learned my addr 172.21.15.43:0/258050342 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d1effd700 1 -- 172.21.15.43:0/258050342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10009b30 con 0x7f9d20106550 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d1effd700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d2019ba60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9d10003c50 tx=0x7f9d10003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d17fff700 1 -- 172.21.15.43:0/258050342 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d1000e040 con 0x7f9d20106550 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d17fff700 1 -- 172.21.15.43:0/258050342 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f9d100094d0 con 0x7f9d20106550 2024-02-05T07:29:02.268 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2019c1a0 con 0x7f9d20106550 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d17fff700 1 -- 172.21.15.43:0/258050342 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d100127c0 con 0x7f9d20106550 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.014+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2019c6a0 con 0x7f9d20106550 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.015+0000 7f9d17fff700 1 -- 172.21.15.43:0/258050342 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f9d10012970 con 0x7f9d20106550 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.015+0000 7f9d17fff700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f9d0c038ee0 0x7f9d0c03b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.015+0000 7f9d17fff700 1 -- 172.21.15.43:0/258050342 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9d1004d3e0 con 0x7f9d20106550 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.015+0000 7f9d1e7fc700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f9d0c038ee0 0x7f9d0c03b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.015+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d20063350 con 0x7f9d20106550 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.016+0000 7f9d1e7fc700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f9d0c038ee0 0x7f9d0c03b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d08006fd0 tx=0x7f9d08006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.020+0000 7f9d17fff700 1 -- 172.21.15.43:0/258050342 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d10020080 con 0x7f9d20106550 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.239+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f9d20101090 con 0x7f9d0c038ee0 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.244+0000 7f9d17fff700 1 -- 172.21.15.43:0/258050342 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9d20101090 con 0x7f9d0c038ee0 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.246+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f9d0c038ee0 msgr2=0x7f9d0c03b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:02.269 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f9d0c038ee0 0x7f9d0c03b3a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9d08006fd0 tx=0x7f9d08006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 msgr2=0x7f9d2019ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d2019ba60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9d10003c50 tx=0x7f9d10003d30 comp rx=0 tx=0).stop 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 shutdown_connections 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f9d0c038ee0 0x7f9d0c03b3a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 --2- 172.21.15.43:0/258050342 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d20106550 0x7f9d2019ba60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 >> 172.21.15.43:0/258050342 conn(0x7f9d200fff80 msgr2=0x7f9d20100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 shutdown_connections 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:01.247+0000 7f9d254b3700 1 -- 172.21.15.43:0/258050342 wait complete. 2024-02-05T07:29:02.270 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-02-05T07:29:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:02 smithi043 ceph-mon[29879]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:02 smithi043 ceph-mon[29879]: Saving service mon spec with placement count:5 2024-02-05T07:29:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:02 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:29:03.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-05T07:29:03.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.671+0000 7f750d0b8700 1 Processor -- start 2024-02-05T07:29:03.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.671+0000 7f750d0b8700 1 -- start start 2024-02-05T07:29:03.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f750d0b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:03.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f750d0b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7508106470 con 0x7f7508105a80 2024-02-05T07:29:03.683 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7506d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7506d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43690/0 (socket says 172.21.15.43:43690) 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7506d9d700 1 -- 172.21.15.43:0/2359415617 learned_addr learned my addr 172.21.15.43:0/2359415617 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7506d9d700 1 -- 172.21.15.43:0/2359415617 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508106c80 con 0x7f7508105a80 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7506d9d700 1 --2- 172.21.15.43:0/2359415617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508105ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f74f00098a0 tx=0x7f74f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f77a46e71bac76e server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7505d9b700 1 -- 172.21.15.43:0/2359415617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f0004160 con 0x7f7508105a80 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7505d9b700 1 -- 172.21.15.43:0/2359415617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f74f00042c0 con 0x7f7508105a80 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.672+0000 7f7505d9b700 1 -- 172.21.15.43:0/2359415617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f00118a0 con 0x7f7508105a80 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.673+0000 7f750d0b8700 1 -- 172.21.15.43:0/2359415617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 msgr2=0x7f7508105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.673+0000 7f750d0b8700 1 --2- 172.21.15.43:0/2359415617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508105ea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f74f00098a0 tx=0x7f74f0009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.673+0000 7f750d0b8700 1 -- 172.21.15.43:0/2359415617 shutdown_connections 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.673+0000 7f750d0b8700 1 --2- 172.21.15.43:0/2359415617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508105ea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.673+0000 7f750d0b8700 1 -- 172.21.15.43:0/2359415617 >> 172.21.15.43:0/2359415617 conn(0x7f7508101040 msgr2=0x7f7508103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.673+0000 7f750d0b8700 1 -- 172.21.15.43:0/2359415617 shutdown_connections 2024-02-05T07:29:03.684 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.673+0000 7f750d0b8700 1 -- 172.21.15.43:0/2359415617 wait complete. 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f750d0b8700 1 Processor -- start 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f750d0b8700 1 -- start start 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f750d0b8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f750d0b8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7508199c60 con 0x7f7508105a80 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f7506d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f7506d9d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43692/0 (socket says 172.21.15.43:43692) 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f7506d9d700 1 -- 172.21.15.43:0/4101612792 learned_addr learned my addr 172.21.15.43:0/4101612792 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.674+0000 7f7506d9d700 1 -- 172.21.15.43:0/4101612792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f0009550 con 0x7f7508105a80 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.675+0000 7f7506d9d700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508199720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74f0003f10 tx=0x7f74f0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.675+0000 7f74fffff700 1 -- 172.21.15.43:0/4101612792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f00036a0 con 0x7f7508105a80 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.675+0000 7f74fffff700 1 -- 172.21.15.43:0/4101612792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f74f001a430 con 0x7f7508105a80 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.675+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7508199e60 con 0x7f7508105a80 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.675+0000 7f74fffff700 1 -- 172.21.15.43:0/4101612792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74f0024410 con 0x7f7508105a80 2024-02-05T07:29:03.685 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.675+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f750819a300 con 0x7f7508105a80 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.676+0000 7f74fffff700 1 -- 172.21.15.43:0/4101612792 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f74f002d3f0 con 0x7f7508105a80 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.676+0000 7f74fffff700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f74f4038f30 0x7f74f403b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.676+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7508063350 con 0x7f7508105a80 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.676+0000 7f74fffff700 1 -- 172.21.15.43:0/4101612792 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f74f004cd40 con 0x7f7508105a80 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.676+0000 7f750659c700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f74f4038f30 0x7f74f403b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.676+0000 7f750659c700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f74f4038f30 0x7f74f403b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f74f8006fd0 tx=0x7f74f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.681+0000 7f74fffff700 1 -- 172.21.15.43:0/4101612792 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74f0010970 con 0x7f7508105a80 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.900+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7508102240 con 0x7f74f4038f30 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.904+0000 7f74fffff700 1 -- 172.21.15.43:0/4101612792 <== mgr.14120 v2:172.21.15.43:6800/4155203878 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7508102240 con 0x7f74f4038f30 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f74f4038f30 msgr2=0x7f74f403b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f74f4038f30 0x7f74f403b3f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f74f8006fd0 tx=0x7f74f8006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 msgr2=0x7f7508199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508199720 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f74f0003f10 tx=0x7f74f0011940 comp rx=0 tx=0).stop 2024-02-05T07:29:03.686 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 shutdown_connections 2024-02-05T07:29:03.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f74f4038f30 0x7f74f403b3f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:03.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 --2- 172.21.15.43:0/4101612792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7508105a80 0x7f7508199720 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:03.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 >> 172.21.15.43:0/4101612792 conn(0x7f7508101040 msgr2=0x7f7508101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:03.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.906+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 shutdown_connections 2024-02-05T07:29:03.687 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:02.907+0000 7f750d0b8700 1 -- 172.21.15.43:0/4101612792 wait complete. 2024-02-05T07:29:04.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:03 smithi043 ceph-mon[29879]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:03 smithi043 ceph-mon[29879]: Saving service mgr spec with placement count:2 2024-02-05T07:29:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:03 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:29:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:03 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:29:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:03 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:29:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3759476108' entity='client.admin' 2024-02-05T07:29:05.617 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.151+0000 7ffb7b35d700 1 Processor -- start 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.151+0000 7ffb7b35d700 1 -- start start 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.151+0000 7ffb7b35d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb741041a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.151+0000 7ffb7b35d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74104770 con 0x7ffb74103d80 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb790f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb741041a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb790f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb741041a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43704/0 (socket says 172.21.15.43:43704) 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb790f9700 1 -- 172.21.15.43:0/4288739912 learned_addr learned my addr 172.21.15.43:0/4288739912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb790f9700 1 -- 172.21.15.43:0/4288739912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74106d10 con 0x7ffb74103d80 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb790f9700 1 --2- 172.21.15.43:0/4288739912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb741041a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb640098a0 tx=0x7ffb64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8502202c89a0e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb6bfff700 1 -- 172.21.15.43:0/4288739912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb64004160 con 0x7ffb74103d80 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb6bfff700 1 -- 172.21.15.43:0/4288739912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffb640042c0 con 0x7ffb74103d80 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.152+0000 7ffb6bfff700 1 -- 172.21.15.43:0/4288739912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb64011850 con 0x7ffb74103d80 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.153+0000 7ffb7b35d700 1 -- 172.21.15.43:0/4288739912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 msgr2=0x7ffb741041a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.153+0000 7ffb7b35d700 1 --2- 172.21.15.43:0/4288739912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb741041a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb640098a0 tx=0x7ffb64009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.153+0000 7ffb7b35d700 1 -- 172.21.15.43:0/4288739912 shutdown_connections 2024-02-05T07:29:05.618 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.153+0000 7ffb7b35d700 1 --2- 172.21.15.43:0/4288739912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb741041a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.153+0000 7ffb7b35d700 1 -- 172.21.15.43:0/4288739912 >> 172.21.15.43:0/4288739912 conn(0x7ffb740ff750 msgr2=0x7ffb74101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.153+0000 7ffb7b35d700 1 -- 172.21.15.43:0/4288739912 shutdown_connections 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.153+0000 7ffb7b35d700 1 -- 172.21.15.43:0/4288739912 wait complete. 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb7b35d700 1 Processor -- start 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb7b35d700 1 -- start start 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb7b35d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb74106590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb7b35d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74106ad0 con 0x7ffb74103d80 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb790f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb74106590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb790f9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb74106590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43720/0 (socket says 172.21.15.43:43720) 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb790f9700 1 -- 172.21.15.43:0/3759476108 learned_addr learned my addr 172.21.15.43:0/3759476108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb790f9700 1 -- 172.21.15.43:0/3759476108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb64009550 con 0x7ffb74103d80 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.154+0000 7ffb790f9700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb74106590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb64006af0 tx=0x7ffb64011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.155+0000 7ffb6a7fc700 1 -- 172.21.15.43:0/3759476108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb640036a0 con 0x7ffb74103d80 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.155+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb74104cc0 con 0x7ffb74103d80 2024-02-05T07:29:05.619 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.155+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb74105160 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.155+0000 7ffb6a7fc700 1 -- 172.21.15.43:0/3759476108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffb6401a430 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.155+0000 7ffb6a7fc700 1 -- 172.21.15.43:0/3759476108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb640193c0 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.155+0000 7ffb6a7fc700 1 -- 172.21.15.43:0/3759476108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ffb64019520 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.156+0000 7ffb6a7fc700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ffb60038ee0 0x7ffb6003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.156+0000 7ffb6a7fc700 1 -- 172.21.15.43:0/3759476108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffb6404dab0 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.156+0000 7ffb788f8700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ffb60038ee0 0x7ffb6003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.156+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb58005150 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.156+0000 7ffb788f8700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ffb60038ee0 0x7ffb6003b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb70006fd0 tx=0x7ffb70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.161+0000 7ffb6a7fc700 1 -- 172.21.15.43:0/3759476108 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb6402d430 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.404+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ffb58005400 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.406+0000 7ffb6a7fc700 1 -- 172.21.15.43:0/3759476108 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ffb64023c20 con 0x7ffb74103d80 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ffb60038ee0 msgr2=0x7ffb6003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ffb60038ee0 0x7ffb6003b3a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ffb70006fd0 tx=0x7ffb70006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:05.620 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 msgr2=0x7ffb74106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:05.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb74106590 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb64006af0 tx=0x7ffb64011a50 comp rx=0 tx=0).stop 2024-02-05T07:29:05.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 shutdown_connections 2024-02-05T07:29:05.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ffb60038ee0 0x7ffb6003b3a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:05.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 --2- 172.21.15.43:0/3759476108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb74103d80 0x7ffb74106590 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:05.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 >> 172.21.15.43:0/3759476108 conn(0x7ffb740ff750 msgr2=0x7ffb74100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:05.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 shutdown_connections 2024-02-05T07:29:05.621 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:04.408+0000 7ffb7b35d700 1 -- 172.21.15.43:0/3759476108 wait complete. 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.114+0000 7ff64e59e700 1 Processor -- start 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.114+0000 7ff64e59e700 1 -- start start 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.114+0000 7ff64e59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff648104db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.114+0000 7ff64e59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648105380 con 0x7ff648104990 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.114+0000 7ff647fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff648104db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.114+0000 7ff647fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff648104db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43734/0 (socket says 172.21.15.43:43734) 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.114+0000 7ff647fff700 1 -- 172.21.15.43:0/3764295442 learned_addr learned my addr 172.21.15.43:0/3764295442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.115+0000 7ff647fff700 1 -- 172.21.15.43:0/3764295442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff648105b90 con 0x7ff648104990 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.115+0000 7ff647fff700 1 --2- 172.21.15.43:0/3764295442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff648104db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098a0 tx=0x7ff638009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3e70003b5f7301 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.115+0000 7ff646ffd700 1 -- 172.21.15.43:0/3764295442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff638004160 con 0x7ff648104990 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.115+0000 7ff646ffd700 1 -- 172.21.15.43:0/3764295442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff6380042c0 con 0x7ff648104990 2024-02-05T07:29:06.908 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.115+0000 7ff646ffd700 1 -- 172.21.15.43:0/3764295442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff638011850 con 0x7ff648104990 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.116+0000 7ff64e59e700 1 -- 172.21.15.43:0/3764295442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 msgr2=0x7ff648104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.116+0000 7ff64e59e700 1 --2- 172.21.15.43:0/3764295442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff648104db0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098a0 tx=0x7ff638009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.116+0000 7ff64e59e700 1 -- 172.21.15.43:0/3764295442 shutdown_connections 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.116+0000 7ff64e59e700 1 --2- 172.21.15.43:0/3764295442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff648104db0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.116+0000 7ff64e59e700 1 -- 172.21.15.43:0/3764295442 >> 172.21.15.43:0/3764295442 conn(0x7ff6480fffd0 msgr2=0x7ff6481023f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.116+0000 7ff64e59e700 1 -- 172.21.15.43:0/3764295442 shutdown_connections 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.116+0000 7ff64e59e700 1 -- 172.21.15.43:0/3764295442 wait complete. 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.117+0000 7ff64e59e700 1 Processor -- start 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.117+0000 7ff64e59e700 1 -- start start 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.117+0000 7ff64e59e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff6481975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.117+0000 7ff64e59e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff648197af0 con 0x7ff648104990 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.117+0000 7ff647fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff6481975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff647fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff6481975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43736/0 (socket says 172.21.15.43:43736) 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff647fff700 1 -- 172.21.15.43:0/2577023034 learned_addr learned my addr 172.21.15.43:0/2577023034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff647fff700 1 -- 172.21.15.43:0/2577023034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638009550 con 0x7ff648104990 2024-02-05T07:29:06.909 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff647fff700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff6481975b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff638003d50 tx=0x7ff638011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff6457fa700 1 -- 172.21.15.43:0/2577023034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6380037d0 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff648197cf0 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff648198190 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff6457fa700 1 -- 172.21.15.43:0/2577023034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff638003930 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.118+0000 7ff6457fa700 1 -- 172.21.15.43:0/2577023034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff6380193c0 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.119+0000 7ff6457fa700 1 -- 172.21.15.43:0/2577023034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7ff638019630 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.119+0000 7ff6457fa700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff630038fd0 0x7ff63003b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.119+0000 7ff6457fa700 1 -- 172.21.15.43:0/2577023034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff63804de00 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.120+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff648063350 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.120+0000 7ff6477fe700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff630038fd0 0x7ff63003b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.120+0000 7ff6477fe700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff630038fd0 0x7ff63003b490 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006fd0 tx=0x7ff63c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.125+0000 7ff6457fa700 1 -- 172.21.15.43:0/2577023034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff63802c3a0 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.382+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ff648190c90 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.384+0000 7ff6457fa700 1 -- 172.21.15.43:0/2577023034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ff638010380 con 0x7ff648104990 2024-02-05T07:29:06.910 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.386+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff630038fd0 msgr2=0x7ff63003b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.386+0000 7ff64e59e700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff630038fd0 0x7ff63003b490 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006fd0 tx=0x7ff63c006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 msgr2=0x7ff6481975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff6481975b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff638003d50 tx=0x7ff638011ac0 comp rx=0 tx=0).stop 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 shutdown_connections 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff630038fd0 0x7ff63003b490 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 --2- 172.21.15.43:0/2577023034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff648104990 0x7ff6481975b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 >> 172.21.15.43:0/2577023034 conn(0x7ff6480fffd0 msgr2=0x7ff64818dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 shutdown_connections 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:06.387+0000 7ff64e59e700 1 -- 172.21.15.43:0/2577023034 wait complete. 2024-02-05T07:29:06.911 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-02-05T07:29:07.018 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:06 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:29:07.018 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2577023034' entity='client.admin' 2024-02-05T07:29:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:07 smithi043 ceph-mon[29879]: from='mgr.14120 172.21.15.43:0/650229396' entity='mgr.y' 2024-02-05T07:29:08.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1367865285' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-05T07:29:08.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:07 smithi043 ceph-mon[29879]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:08.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:08 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ignoring --setuser ceph since I am not root 2024-02-05T07:29:08.314 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:08 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ignoring --setgroup ceph since I am not root 2024-02-05T07:29:08.744 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:08 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:08.552+0000 7fb7c341f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:29:08.744 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:08 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:08.647+0000 7fb7c341f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:29:08.747 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.423+0000 7f8c5e9da700 1 Processor -- start 2024-02-05T07:29:08.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.423+0000 7f8c5e9da700 1 -- start start 2024-02-05T07:29:08.752 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.424+0000 7f8c5e9da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c58104780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.424+0000 7f8c5e9da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c580796c0 con 0x7f8c58104360 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.424+0000 7f8c57fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c58104780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.424+0000 7f8c57fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c58104780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43750/0 (socket says 172.21.15.43:43750) 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.424+0000 7f8c57fff700 1 -- 172.21.15.43:0/1328538643 learned_addr learned my addr 172.21.15.43:0/1328538643 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.424+0000 7f8c57fff700 1 -- 172.21.15.43:0/1328538643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c58104cc0 con 0x7f8c58104360 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.424+0000 7f8c57fff700 1 --2- 172.21.15.43:0/1328538643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c58104780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c480098a0 tx=0x7f8c48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290858e021e4fe23 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.425+0000 7f8c56ffd700 1 -- 172.21.15.43:0/1328538643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c48004160 con 0x7f8c58104360 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.425+0000 7f8c56ffd700 1 -- 172.21.15.43:0/1328538643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8c480042c0 con 0x7f8c58104360 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.425+0000 7f8c56ffd700 1 -- 172.21.15.43:0/1328538643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c48011850 con 0x7f8c58104360 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.425+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1328538643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 msgr2=0x7f8c58104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.425+0000 7f8c5e9da700 1 --2- 172.21.15.43:0/1328538643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c58104780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8c480098a0 tx=0x7f8c48009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.426+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1328538643 shutdown_connections 2024-02-05T07:29:08.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.426+0000 7f8c5e9da700 1 --2- 172.21.15.43:0/1328538643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c58104780 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.426+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1328538643 >> 172.21.15.43:0/1328538643 conn(0x7f8c580fff80 msgr2=0x7f8c581023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.426+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1328538643 shutdown_connections 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.426+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1328538643 wait complete. 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.426+0000 7f8c5e9da700 1 Processor -- start 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.426+0000 7f8c5e9da700 1 -- start start 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c5e9da700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c5819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c5e9da700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5819be40 con 0x7f8c58104360 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c57fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c5819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c57fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c5819b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43760/0 (socket says 172.21.15.43:43760) 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c57fff700 1 -- 172.21.15.43:0/1367865285 learned_addr learned my addr 172.21.15.43:0/1367865285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c57fff700 1 -- 172.21.15.43:0/1367865285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c48009550 con 0x7f8c58104360 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c57fff700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c5819b900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c48003ec0 tx=0x7f8c48011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c480037d0 con 0x7f8c58104360 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8c48003930 con 0x7f8c58104360 2024-02-05T07:29:08.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.427+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5819c040 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.428+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8c480193c0 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.428+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5819c4e0 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.428+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 8) v1 ==== 45932+0+0 (secure 0 0 0) 0x7f8c48019650 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.428+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c44005150 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.429+0000 7f8c557fa700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f8c40038ee0 0x7f8c4003b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.429+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8c4801a8b0 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.429+0000 7f8c577fe700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f8c40038ee0 0x7f8c4003b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.429+0000 7f8c577fe700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f8c40038ee0 0x7f8c4003b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006fd0 tx=0x7f8c4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.434+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c4800b3e0 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.708+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8c44005400 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.971+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8c48018350 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.973+0000 7f8c557fa700 1 -- 172.21.15.43:0/1367865285 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f8c4800b760 con 0x7f8c58104360 2024-02-05T07:29:08.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f8c40038ee0 msgr2=0x7f8c4003b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f8c40038ee0 0x7f8c4003b3a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c006fd0 tx=0x7f8c4c006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 msgr2=0x7f8c5819b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c5819b900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8c48003ec0 tx=0x7f8c48011980 comp rx=0 tx=0).stop 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 shutdown_connections 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7f8c40038ee0 0x7f8c4003b3a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 --2- 172.21.15.43:0/1367865285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c58104360 0x7f8c5819b900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 >> 172.21.15.43:0/1367865285 conn(0x7f8c580fff80 msgr2=0x7f8c58192780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 shutdown_connections 2024-02-05T07:29:08.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:07.974+0000 7f8c5e9da700 1 -- 172.21.15.43:0/1367865285 wait complete. 2024-02-05T07:29:09.055 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1367865285' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-05T07:29:09.056 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:08 smithi043 ceph-mon[29879]: mgrmap e9: y(active, since 20s) 2024-02-05T07:29:09.585 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:09.339+0000 7fb7c341f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:29:09.586 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:09.435+0000 7fb7c341f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:29:09.586 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:09.584+0000 7fb7c341f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:29:09.902 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:09.683+0000 7fb7c341f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:29:09.903 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:09.768+0000 7fb7c341f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:29:10.153 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/906674117' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-05T07:29:10.153 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:10 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:10.042+0000 7fb7c341f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:29:10.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:29:10.160 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.159+0000 7ff956444700 1 Processor -- start 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.159+0000 7ff956444700 1 -- start start 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.159+0000 7ff956444700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff950105ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.159+0000 7ff956444700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950106470 con 0x7ff950105a80 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.159+0000 7ff94ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff950105ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.159+0000 7ff94ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff950105ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43778/0 (socket says 172.21.15.43:43778) 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.159+0000 7ff94ffff700 1 -- 172.21.15.43:0/3074698725 learned_addr learned my addr 172.21.15.43:0/3074698725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.160+0000 7ff94ffff700 1 -- 172.21.15.43:0/3074698725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff950106c80 con 0x7ff950105a80 2024-02-05T07:29:10.161 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.160+0000 7ff94ffff700 1 --2- 172.21.15.43:0/3074698725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff950105ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9400098a0 tx=0x7ff940009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573921f9f2e82055 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.160+0000 7ff94effd700 1 -- 172.21.15.43:0/3074698725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff940004160 con 0x7ff950105a80 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.160+0000 7ff94effd700 1 -- 172.21.15.43:0/3074698725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff9400042c0 con 0x7ff950105a80 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.160+0000 7ff94effd700 1 -- 172.21.15.43:0/3074698725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff940011850 con 0x7ff950105a80 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 -- 172.21.15.43:0/3074698725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 msgr2=0x7ff950105ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 --2- 172.21.15.43:0/3074698725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff950105ea0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff9400098a0 tx=0x7ff940009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 -- 172.21.15.43:0/3074698725 shutdown_connections 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 --2- 172.21.15.43:0/3074698725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff950105ea0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 -- 172.21.15.43:0/3074698725 >> 172.21.15.43:0/3074698725 conn(0x7ff950101040 msgr2=0x7ff950103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 -- 172.21.15.43:0/3074698725 shutdown_connections 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 -- 172.21.15.43:0/3074698725 wait complete. 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.161+0000 7ff956444700 1 Processor -- start 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff956444700 1 -- start start 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff956444700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff9501996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff956444700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950199c30 con 0x7ff950105a80 2024-02-05T07:29:10.162 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff94ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff9501996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff94ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff9501996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43794/0 (socket says 172.21.15.43:43794) 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff94ffff700 1 -- 172.21.15.43:0/906674117 learned_addr learned my addr 172.21.15.43:0/906674117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff94ffff700 1 -- 172.21.15.43:0/906674117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940009550 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.162+0000 7ff94ffff700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff9501996f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff940003d50 tx=0x7ff940011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.163+0000 7ff94d7fa700 1 -- 172.21.15.43:0/906674117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9400037d0 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.163+0000 7ff94d7fa700 1 -- 172.21.15.43:0/906674117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff940003930 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.163+0000 7ff94d7fa700 1 -- 172.21.15.43:0/906674117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff9400193c0 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.163+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff950199e30 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.163+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95019a2d0 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.163+0000 7ff94d7fa700 1 -- 172.21.15.43:0/906674117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7ff940019680 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.164+0000 7ff94d7fa700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 0x7ff93c03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.164+0000 7ff94d7fa700 1 -- 172.21.15.43:0/906674117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff94004cfc0 con 0x7ff950105a80 2024-02-05T07:29:10.163 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.164+0000 7ff94f7fe700 1 -- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 msgr2=0x7ff93c03b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4155203878 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.164+0000 7ff94f7fe700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 0x7ff93c03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.164+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff950063350 con 0x7ff950105a80 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.169+0000 7ff94d7fa700 1 -- 172.21.15.43:0/906674117 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9400199d0 con 0x7ff950105a80 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.365+0000 7ff94f7fe700 1 -- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 msgr2=0x7ff93c03b3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4155203878 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.365+0000 7ff94f7fe700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 0x7ff93c03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.458+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff95019cc10 con 0x7ff950105a80 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.459+0000 7ff94d7fa700 1 -- 172.21.15.43:0/906674117 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7ff940010ed0 con 0x7ff950105a80 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 msgr2=0x7ff93c03b3f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 0x7ff93c03b3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 msgr2=0x7ff9501996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff9501996f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff940003d50 tx=0x7ff940011ac0 comp rx=0 tx=0).stop 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 shutdown_connections 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7ff93c038f30 0x7ff93c03b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:10.164 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 --2- 172.21.15.43:0/906674117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950105a80 0x7ff9501996f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:10.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 >> 172.21.15.43:0/906674117 conn(0x7ff950101040 msgr2=0x7ff950101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:10.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 shutdown_connections 2024-02-05T07:29:10.165 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:09.461+0000 7ff956444700 1 -- 172.21.15.43:0/906674117 wait complete. 2024-02-05T07:29:10.165 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-02-05T07:29:10.165 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-02-05T07:29:10.489 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:10 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:10.156+0000 7fb7c341f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:29:10.489 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:10 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:10.249+0000 7fb7c341f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:29:10.489 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:10 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:10.347+0000 7fb7c341f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:29:10.799 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:10 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:10.533+0000 7fb7c341f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:29:11.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:11 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:11.531+0000 7fb7c341f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:29:11.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:11 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:11.665+0000 7fb7c341f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:29:12.300 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:12 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:12.005+0000 7fb7c341f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:29:12.300 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:12 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:12.098+0000 7fb7c341f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:29:12.300 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:12 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:12.298+0000 7fb7c341f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:29:12.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:12 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:12.487+0000 7fb7c341f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:29:13.043 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:12 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:12.779+0000 7fb7c341f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:29:13.043 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:13 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:13.041+0000 7fb7c341f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:29:13.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:13 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:13.283+0000 7fb7c341f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:29:14.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:14 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:14.111+0000 7fb7c341f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:29:15.019 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:14 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:14.694+0000 7fb7c341f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:29:15.019 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:14 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:14.795+0000 7fb7c341f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:29:15.019 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:29:15.017+0000 7fb7c341f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: Active manager daemon y restarted 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: Activating manager daemon y 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: osdmap e3: 0 total, 0 up, 0 in 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: mgrmap e10: y(active, starting, since 0.00324798s) 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:29:16.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:29:16.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:29:16.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: Manager daemon y is now available 2024-02-05T07:29:16.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:16.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:16.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:29:16.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:29:16.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-02-05T07:29:16.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-05T07:29:16.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-05T07:29:16.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-02-05T07:29:16.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.591+0000 7fc4b42e4700 1 Processor -- start 2024-02-05T07:29:16.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.591+0000 7fc4b42e4700 1 -- start start 2024-02-05T07:29:16.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.592+0000 7fc4b42e4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.592+0000 7fc4b42e4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac105390 con 0x7fc4ac1049a0 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.592+0000 7fc4b2080700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.592+0000 7fc4b2080700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43806/0 (socket says 172.21.15.43:43806) 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.592+0000 7fc4b2080700 1 -- 172.21.15.43:0/4225199711 learned_addr learned my addr 172.21.15.43:0/4225199711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.592+0000 7fc4b2080700 1 -- 172.21.15.43:0/4225199711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4ac105ba0 con 0x7fc4ac1049a0 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.592+0000 7fc4b2080700 1 --2- 172.21.15.43:0/4225199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac104dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00098a0 tx=0x7fc4a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0ab91481cfaeed server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b107e700 1 -- 172.21.15.43:0/4225199711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4a0004160 con 0x7fc4ac1049a0 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b107e700 1 -- 172.21.15.43:0/4225199711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc4a00042c0 con 0x7fc4ac1049a0 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b107e700 1 -- 172.21.15.43:0/4225199711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4a0011850 con 0x7fc4ac1049a0 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b42e4700 1 -- 172.21.15.43:0/4225199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 msgr2=0x7fc4ac104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b42e4700 1 --2- 172.21.15.43:0/4225199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac104dc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc4a00098a0 tx=0x7fc4a0009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b42e4700 1 -- 172.21.15.43:0/4225199711 shutdown_connections 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b42e4700 1 --2- 172.21.15.43:0/4225199711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac104dc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:16.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b42e4700 1 -- 172.21.15.43:0/4225199711 >> 172.21.15.43:0/4225199711 conn(0x7fc4ac0fffa0 msgr2=0x7fc4ac1023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b42e4700 1 -- 172.21.15.43:0/4225199711 shutdown_connections 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.593+0000 7fc4b42e4700 1 -- 172.21.15.43:0/4225199711 wait complete. 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.594+0000 7fc4b42e4700 1 Processor -- start 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.594+0000 7fc4b42e4700 1 -- start start 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.594+0000 7fc4b42e4700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.594+0000 7fc4b42e4700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4ac19bec0 con 0x7fc4ac1049a0 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.594+0000 7fc4b2080700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.594+0000 7fc4b2080700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43818/0 (socket says 172.21.15.43:43818) 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.594+0000 7fc4b2080700 1 -- 172.21.15.43:0/4034759126 learned_addr learned my addr 172.21.15.43:0/4034759126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.595+0000 7fc4b2080700 1 -- 172.21.15.43:0/4034759126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a0009550 con 0x7fc4ac1049a0 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.595+0000 7fc4b2080700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac19b980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006af0 tx=0x7fc4a0011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.595+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4a00036a0 con 0x7fc4ac1049a0 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.595+0000 7fc4b42e4700 1 -- 172.21.15.43:0/4034759126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4ac19c0c0 con 0x7fc4ac1049a0 2024-02-05T07:29:16.673 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.595+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc4a001a430 con 0x7fc4ac1049a0 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.595+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc4a00193c0 con 0x7fc4ac1049a0 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.595+0000 7fc4b42e4700 1 -- 172.21.15.43:0/4034759126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4ac19c560 con 0x7fc4ac1049a0 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.596+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 9) v1 ==== 45945+0+0 (secure 0 0 0) 0x7fc4a0019630 con 0x7fc4ac1049a0 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.596+0000 7fc49f7fe700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.596+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 --> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4ac063350 con 0x7fc498038ee0 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.596+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc4a0003800 con 0x7fc4ac1049a0 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.596+0000 7fc4b187f700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 msgr2=0x7fc49803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4155203878 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.596+0000 7fc4b187f700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.797+0000 7fc4b187f700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 msgr2=0x7fc49803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4155203878 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:10.797+0000 7fc4b187f700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:11.198+0000 7fc4b187f700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 msgr2=0x7fc49803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4155203878 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:11.198+0000 7fc4b187f700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:11.999+0000 7fc4b187f700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 msgr2=0x7fc49803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4155203878 2024-02-05T07:29:16.674 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:11.999+0000 7fc4b187f700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:13.600+0000 7fc4b187f700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 msgr2=0x7fc49803b3a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/4155203878 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:13.600+0000 7fc4b187f700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:15.033+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mgrmap(e 10) v1 ==== 45712+0+0 (secure 0 0 0) 0x7fc4a002d950 con 0x7fc4ac1049a0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:15.033+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 msgr2=0x7fc49803b3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:15.033+0000 7fc49f7fe700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.034+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7fc4a002ad20 con 0x7fc4ac1049a0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.034+0000 7fc49f7fe700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fc49803c7a0 0x7fc49803eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.035+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc4ac063350 con 0x7fc49803c7a0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.035+0000 7fc4b187f700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fc49803c7a0 0x7fc49803eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.036+0000 7fc4b187f700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fc49803c7a0 0x7fc49803eb90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8003a10 tx=0x7fc4a80092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.036+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7fc4ac063350 con 0x7fc49803c7a0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.043+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fc4ac108dd0 con 0x7fc49803c7a0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.043+0000 7fc49f7fe700 1 -- 172.21.15.43:0/4034759126 <== mgr.14152 v2:172.21.15.43:6800/629017799 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fc4ac108dd0 con 0x7fc49803c7a0 2024-02-05T07:29:16.675 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fc49803c7a0 msgr2=0x7fc49803eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fc49803c7a0 0x7fc49803eb90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8003a10 tx=0x7fc4a80092b0 comp rx=0 tx=0).stop 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 msgr2=0x7fc4ac19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac19b980 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006af0 tx=0x7fc4a0011a50 comp rx=0 tx=0).stop 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 shutdown_connections 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fc49803c7a0 0x7fc49803eb90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4ac1049a0 0x7fc4ac19b980 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 --2- 172.21.15.43:0/4034759126 >> [v2:172.21.15.43:6800/4155203878,v1:172.21.15.43:6801/4155203878] conn(0x7fc498038ee0 0x7fc49803b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 >> 172.21.15.43:0/4034759126 conn(0x7fc4ac0fffa0 msgr2=0x7fc4ac100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 shutdown_connections 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:16.044+0000 7fc49d7fa700 1 -- 172.21.15.43:0/4034759126 wait complete. 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-02-05T07:29:16.676 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-02-05T07:29:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:16 smithi043 ceph-mon[29879]: [05/Feb/2024:07:29:15] ENGINE Bus STARTING 2024-02-05T07:29:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:16 smithi043 ceph-mon[29879]: [05/Feb/2024:07:29:15] ENGINE Serving on https://172.21.15.43:7150 2024-02-05T07:29:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:16 smithi043 ceph-mon[29879]: [05/Feb/2024:07:29:15] ENGINE Serving on http://172.21.15.43:8765 2024-02-05T07:29:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:16 smithi043 ceph-mon[29879]: [05/Feb/2024:07:29:15] ENGINE Bus STARTED 2024-02-05T07:29:16.997 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:16 smithi043 ceph-mon[29879]: mgrmap e11: y(active, since 1.0062s) 2024-02-05T07:29:18.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:17 smithi043 ceph-mon[29879]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-05T07:29:18.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:17 smithi043 ceph-mon[29879]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-05T07:29:18.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:17 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:18.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:17 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:18.286 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:17 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.361+0000 7f54ee294700 1 Processor -- start 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.361+0000 7f54ee294700 1 -- start start 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.361+0000 7f54ee294700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8104dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.361+0000 7f54ee294700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e8105390 con 0x7f54e81049a0 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.361+0000 7f54e7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8104dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.361+0000 7f54e7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8104dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41544/0 (socket says 172.21.15.43:41544) 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.361+0000 7f54e7fff700 1 -- 172.21.15.43:0/970510121 learned_addr learned my addr 172.21.15.43:0/970510121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.362+0000 7f54e7fff700 1 -- 172.21.15.43:0/970510121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8105ba0 con 0x7f54e81049a0 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.362+0000 7f54e7fff700 1 --2- 172.21.15.43:0/970510121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8104dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54d80098a0 tx=0x7f54d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f0017e4a247f25 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.362+0000 7f54e6ffd700 1 -- 172.21.15.43:0/970510121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d8004160 con 0x7f54e81049a0 2024-02-05T07:29:18.294 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.362+0000 7f54e6ffd700 1 -- 172.21.15.43:0/970510121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f54d80042c0 con 0x7f54e81049a0 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.362+0000 7f54e6ffd700 1 -- 172.21.15.43:0/970510121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d8011850 con 0x7f54e81049a0 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 -- 172.21.15.43:0/970510121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 msgr2=0x7f54e8104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 --2- 172.21.15.43:0/970510121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8104dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f54d80098a0 tx=0x7f54d8009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 -- 172.21.15.43:0/970510121 shutdown_connections 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 --2- 172.21.15.43:0/970510121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8104dc0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 -- 172.21.15.43:0/970510121 >> 172.21.15.43:0/970510121 conn(0x7f54e80fffa0 msgr2=0x7f54e81023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 -- 172.21.15.43:0/970510121 shutdown_connections 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 -- 172.21.15.43:0/970510121 wait complete. 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 Processor -- start 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.363+0000 7f54ee294700 1 -- start start 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54ee294700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54ee294700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e8197aa0 con 0x7f54e81049a0 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54e7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54e7fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8197560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41558/0 (socket says 172.21.15.43:41558) 2024-02-05T07:29:18.295 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54e7fff700 1 -- 172.21.15.43:0/200298327 learned_addr learned my addr 172.21.15.43:0/200298327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54e7fff700 1 -- 172.21.15.43:0/200298327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d8009550 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54e7fff700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8197560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54d8006af0 tx=0x7f54d8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54e57fa700 1 -- 172.21.15.43:0/200298327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d80036a0 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e8197ca0 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.364+0000 7f54e57fa700 1 -- 172.21.15.43:0/200298327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f54d801a430 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.365+0000 7f54e57fa700 1 -- 172.21.15.43:0/200298327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f54d80193c0 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.365+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e8198140 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.366+0000 7f54e57fa700 1 -- 172.21.15.43:0/200298327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 11) v1 ==== 45839+0+0 (secure 0 0 0) 0x7f54d8019630 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.366+0000 7f54e57fa700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f54d003afd0 0x7f54d003d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.366+0000 7f54e57fa700 1 -- 172.21.15.43:0/200298327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f54d804d0e0 con 0x7f54e81049a0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.366+0000 7f54e77fe700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f54d003afd0 0x7f54d003d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:18.296 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.366+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e8063350 con 0x7f54e81049a0 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.366+0000 7f54e77fe700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f54d003afd0 0x7f54d003d490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54dc006fd0 tx=0x7f54dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.371+0000 7f54e57fa700 1 -- 172.21.15.43:0/200298327 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f54d80199d0 con 0x7f54e81049a0 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.579+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f54e8062160 con 0x7f54d003afd0 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.681+0000 7f54e57fa700 1 -- 172.21.15.43:0/200298327 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f54e8062160 con 0x7f54d003afd0 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f54d003afd0 msgr2=0x7f54d003d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f54d003afd0 0x7f54d003d490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f54dc006fd0 tx=0x7f54dc006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 msgr2=0x7f54e8197560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8197560 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f54d8006af0 tx=0x7f54d8011a50 comp rx=0 tx=0).stop 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 shutdown_connections 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f54d003afd0 0x7f54d003d490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 --2- 172.21.15.43:0/200298327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f54e81049a0 0x7f54e8197560 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 >> 172.21.15.43:0/200298327 conn(0x7f54e80fffa0 msgr2=0x7f54e818dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 shutdown_connections 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:17.684+0000 7f54ee294700 1 -- 172.21.15.43:0/200298327 wait complete. 2024-02-05T07:29:18.297 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-02-05T07:29:19.207 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:18 smithi043 ceph-mon[29879]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:18 smithi043 ceph-mon[29879]: mgrmap e12: y(active, since 2s) 2024-02-05T07:29:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:20.020 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:19 smithi043 ceph-mon[29879]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:20.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$RzL5IIxECAffiA2WlrNWMe0uLCpOV4tsjkU6ZX4tSsOrz7pYR/h.2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707118159, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.745+0000 7f227f344700 1 Processor -- start 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.745+0000 7f227f344700 1 -- start start 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.745+0000 7f227f344700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278108940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.745+0000 7f227f344700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22780796e0 con 0x7f2278106550 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.745+0000 7f227d0e0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.745+0000 7f227d0e0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41562/0 (socket says 172.21.15.43:41562) 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.745+0000 7f227d0e0700 1 -- 172.21.15.43:0/2292822165 learned_addr learned my addr 172.21.15.43:0/2292822165 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.746+0000 7f227d0e0700 1 -- 172.21.15.43:0/2292822165 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2278108e80 con 0x7f2278106550 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.746+0000 7f227d0e0700 1 --2- 172.21.15.43:0/2292822165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278108940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22740098a0 tx=0x7f2274009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238a8cfaac3197aa server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.746+0000 7f226ffff700 1 -- 172.21.15.43:0/2292822165 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2274004160 con 0x7f2278106550 2024-02-05T07:29:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.746+0000 7f226ffff700 1 -- 172.21.15.43:0/2292822165 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f22740042c0 con 0x7f2278106550 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.746+0000 7f226ffff700 1 -- 172.21.15.43:0/2292822165 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2274011850 con 0x7f2278106550 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 -- 172.21.15.43:0/2292822165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 msgr2=0x7f2278108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 --2- 172.21.15.43:0/2292822165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278108940 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f22740098a0 tx=0x7f2274009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 -- 172.21.15.43:0/2292822165 shutdown_connections 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 --2- 172.21.15.43:0/2292822165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278108940 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 -- 172.21.15.43:0/2292822165 >> 172.21.15.43:0/2292822165 conn(0x7f22780fff80 msgr2=0x7f22781023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 -- 172.21.15.43:0/2292822165 shutdown_connections 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 -- 172.21.15.43:0/2292822165 wait complete. 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 Processor -- start 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 -- start start 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278105d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.747+0000 7f227f344700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22781062c0 con 0x7f2278106550 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f227d0e0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278105d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:20.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f227d0e0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278105d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41566/0 (socket says 172.21.15.43:41566) 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f227d0e0700 1 -- 172.21.15.43:0/2452993533 learned_addr learned my addr 172.21.15.43:0/2452993533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f227d0e0700 1 -- 172.21.15.43:0/2452993533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274009550 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f227d0e0700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278105d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2274003ec0 tx=0x7f2274011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f226e7fc700 1 -- 172.21.15.43:0/2452993533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22740037d0 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f226e7fc700 1 -- 172.21.15.43:0/2452993533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2274003930 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f226e7fc700 1 -- 172.21.15.43:0/2452993533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f22740193c0 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2278104500 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.748+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22781049a0 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.749+0000 7f226e7fc700 1 -- 172.21.15.43:0/2452993533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f227401a430 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.749+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2278063350 con 0x7f2278106550 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.749+0000 7f226e7fc700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2264038f90 0x7f226403b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:20.032 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.749+0000 7f226e7fc700 1 -- 172.21.15.43:0/2452993533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f227404d0c0 con 0x7f2278106550 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.750+0000 7f227c8df700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2264038f90 0x7f226403b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.750+0000 7f227c8df700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2264038f90 0x7f226403b450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2268006fd0 tx=0x7f2268006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:18.754+0000 7f226e7fc700 1 -- 172.21.15.43:0/2452993533 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2274003aa0 con 0x7f2278106550 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.014+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2278063170 con 0x7f2264038f90 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.260+0000 7f226e7fc700 1 -- 172.21.15.43:0/2452993533 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f2278063170 con 0x7f2264038f90 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.262+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2264038f90 msgr2=0x7f226403b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.262+0000 7f227f344700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2264038f90 0x7f226403b450 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2268006fd0 tx=0x7f2268006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.262+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 msgr2=0x7f2278105d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.262+0000 7f227f344700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278105d80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2274003ec0 tx=0x7f2274011980 comp rx=0 tx=0).stop 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.263+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 shutdown_connections 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.263+0000 7f227f344700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2264038f90 0x7f226403b450 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.263+0000 7f227f344700 1 --2- 172.21.15.43:0/2452993533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2278106550 0x7f2278105d80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.263+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 >> 172.21.15.43:0/2452993533 conn(0x7f22780fff80 msgr2=0x7f2278100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:20.033 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.263+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 shutdown_connections 2024-02-05T07:29:20.034 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:19.263+0000 7f227f344700 1 -- 172.21.15.43:0/2452993533 wait complete. 2024-02-05T07:29:20.034 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-02-05T07:29:21.371 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2236943267' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-05T07:29:21.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-02-05T07:29:21.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.850+0000 7f243087a700 1 Processor -- start 2024-02-05T07:29:21.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.850+0000 7f243087a700 1 -- start start 2024-02-05T07:29:21.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.850+0000 7f243087a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c1047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:21.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.850+0000 7f243087a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c0796a0 con 0x7f242c104380 2024-02-05T07:29:21.612 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c1047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c1047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41578/0 (socket says 172.21.15.43:41578) 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242a59c700 1 -- 172.21.15.43:0/2615504394 learned_addr learned my addr 172.21.15.43:0/2615504394 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242a59c700 1 -- 172.21.15.43:0/2615504394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c104ce0 con 0x7f242c104380 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242a59c700 1 --2- 172.21.15.43:0/2615504394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c1047a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2424009e80 tx=0x7f242400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c45fa791b9c9b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242959a700 1 -- 172.21.15.43:0/2615504394 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f242400e070 con 0x7f242c104380 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242959a700 1 -- 172.21.15.43:0/2615504394 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f242400b8c0 con 0x7f242c104380 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.851+0000 7f242959a700 1 -- 172.21.15.43:0/2615504394 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24240042d0 con 0x7f242c104380 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.852+0000 7f243087a700 1 -- 172.21.15.43:0/2615504394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 msgr2=0x7f242c1047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.852+0000 7f243087a700 1 --2- 172.21.15.43:0/2615504394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c1047a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2424009e80 tx=0x7f242400b330 comp rx=0 tx=0).stop 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.852+0000 7f243087a700 1 -- 172.21.15.43:0/2615504394 shutdown_connections 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.852+0000 7f243087a700 1 --2- 172.21.15.43:0/2615504394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c1047a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.852+0000 7f243087a700 1 -- 172.21.15.43:0/2615504394 >> 172.21.15.43:0/2615504394 conn(0x7f242c0fff80 msgr2=0x7f242c1023a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.852+0000 7f243087a700 1 -- 172.21.15.43:0/2615504394 shutdown_connections 2024-02-05T07:29:21.613 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.852+0000 7f243087a700 1 -- 172.21.15.43:0/2615504394 wait complete. 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f243087a700 1 Processor -- start 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f243087a700 1 -- start start 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f243087a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c19b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f243087a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c19bde0 con 0x7f242c104380 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f242a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c19b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f242a59c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c19b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41592/0 (socket says 172.21.15.43:41592) 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f242a59c700 1 -- 172.21.15.43:0/2236943267 learned_addr learned my addr 172.21.15.43:0/2236943267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f242a59c700 1 -- 172.21.15.43:0/2236943267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424009b30 con 0x7f242c104380 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f242a59c700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c19b8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f242400b7b0 tx=0x7f24240048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.853+0000 7f24237fe700 1 -- 172.21.15.43:0/2236943267 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f242400e040 con 0x7f242c104380 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.854+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c19bfe0 con 0x7f242c104380 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.854+0000 7f24237fe700 1 -- 172.21.15.43:0/2236943267 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f242400ba30 con 0x7f242c104380 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.854+0000 7f24237fe700 1 -- 172.21.15.43:0/2236943267 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f24240125c0 con 0x7f242c104380 2024-02-05T07:29:21.614 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.854+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c19c4e0 con 0x7f242c104380 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.854+0000 7f24237fe700 1 -- 172.21.15.43:0/2236943267 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 12) v1 ==== 45945+0+0 (secure 0 0 0) 0x7f2424012880 con 0x7f242c104380 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.855+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c195370 con 0x7f242c104380 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.855+0000 7f24237fe700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2418038ee0 0x7f241803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.855+0000 7f24237fe700 1 -- 172.21.15.43:0/2236943267 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f242404d2a0 con 0x7f242c104380 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.855+0000 7f2429d9b700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2418038ee0 0x7f241803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.855+0000 7f2429d9b700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2418038ee0 0x7f241803b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2414006fd0 tx=0x7f2414006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:20.860+0000 7f24237fe700 1 -- 172.21.15.43:0/2236943267 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2424011e70 con 0x7f242c104380 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.062+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f242c105ad0 con 0x7f242c104380 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.063+0000 7f24237fe700 1 -- 172.21.15.43:0/2236943267 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f2424017020 con 0x7f242c104380 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2418038ee0 msgr2=0x7f241803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2418038ee0 0x7f241803b3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2414006fd0 tx=0x7f2414006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 msgr2=0x7f242c19b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c19b8a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f242400b7b0 tx=0x7f24240048c0 comp rx=0 tx=0).stop 2024-02-05T07:29:21.615 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 shutdown_connections 2024-02-05T07:29:21.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2418038ee0 0x7f241803b3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:21.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 --2- 172.21.15.43:0/2236943267 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f242c104380 0x7f242c19b8a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:21.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 >> 172.21.15.43:0/2236943267 conn(0x7f242c0fff80 msgr2=0x7f242c18fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:21.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 shutdown_connections 2024-02-05T07:29:21.616 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:21.065+0000 7f243087a700 1 -- 172.21.15.43:0/2236943267 wait complete. 2024-02-05T07:29:21.619 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-05T07:29:21.619 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-02-05T07:29:21.626 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-05T07:29:21.626 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-02-05T07:29:21.626 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-02-05T07:29:21.626 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-05T07:29:21.628 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-02-05T07:29:21.628 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:21.628 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-02-05T07:29:21.628 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-02-05T07:29:21.629 INFO:teuthology.orchestra.run.smithi043.stdout: Password: 0viia1cly6 2024-02-05T07:29:21.629 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config directory 2024-02-05T07:29:21.629 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling autotune for osd_memory_target 2024-02-05T07:29:22.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:22 smithi043 ceph-mon[29879]: mgrmap e13: y(active, since 6s) 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c8d513700 1 Processor -- start 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c8d513700 1 -- start start 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c8d513700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c88079b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c8d513700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8807a160 con 0x7f6c8807b730 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c88079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c88079b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33862/0 (socket says 172.21.15.43:33862) 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c86ffd700 1 -- 172.21.15.43:0/2359305375 learned_addr learned my addr 172.21.15.43:0/2359305375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c86ffd700 1 -- 172.21.15.43:0/2359305375 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8807a2a0 con 0x7f6c8807b730 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.097+0000 7f6c86ffd700 1 --2- 172.21.15.43:0/2359305375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c88079b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c780098a0 tx=0x7f6c78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799f4168a683685a server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c85ffb700 1 -- 172.21.15.43:0/2359305375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c78004160 con 0x7f6c8807b730 2024-02-05T07:29:22.869 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c85ffb700 1 -- 172.21.15.43:0/2359305375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6c780042c0 con 0x7f6c8807b730 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c85ffb700 1 -- 172.21.15.43:0/2359305375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c78011850 con 0x7f6c8807b730 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c8d513700 1 -- 172.21.15.43:0/2359305375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 msgr2=0x7f6c88079b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c8d513700 1 --2- 172.21.15.43:0/2359305375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c88079b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6c780098a0 tx=0x7f6c78009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c8d513700 1 -- 172.21.15.43:0/2359305375 shutdown_connections 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c8d513700 1 --2- 172.21.15.43:0/2359305375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c88079b90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c8d513700 1 -- 172.21.15.43:0/2359305375 >> 172.21.15.43:0/2359305375 conn(0x7f6c88101070 msgr2=0x7f6c88103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c8d513700 1 -- 172.21.15.43:0/2359305375 shutdown_connections 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.098+0000 7f6c8d513700 1 -- 172.21.15.43:0/2359305375 wait complete. 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c8d513700 1 Processor -- start 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c8d513700 1 -- start start 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c8d513700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c8819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:22.870 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c8d513700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8819be60 con 0x7f6c8807b730 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c8819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c8819b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33870/0 (socket says 172.21.15.43:33870) 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c86ffd700 1 -- 172.21.15.43:0/3793600149 learned_addr learned my addr 172.21.15.43:0/3793600149 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.099+0000 7f6c86ffd700 1 -- 172.21.15.43:0/3793600149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c78009550 con 0x7f6c8807b730 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.100+0000 7f6c86ffd700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c8819b920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c78006af0 tx=0x7f6c78011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.100+0000 7f6c6ffff700 1 -- 172.21.15.43:0/3793600149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c780036a0 con 0x7f6c8807b730 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.100+0000 7f6c6ffff700 1 -- 172.21.15.43:0/3793600149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6c7801a430 con 0x7f6c8807b730 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.100+0000 7f6c6ffff700 1 -- 172.21.15.43:0/3793600149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6c780193c0 con 0x7f6c8807b730 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.100+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8819c060 con 0x7f6c8807b730 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.100+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8819c500 con 0x7f6c8807b730 2024-02-05T07:29:22.871 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.101+0000 7f6c6ffff700 1 -- 172.21.15.43:0/3793600149 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f6c78019520 con 0x7f6c8807b730 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.101+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8810bdd0 con 0x7f6c8807b730 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.101+0000 7f6c6ffff700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f6c70038fe0 0x7f6c7003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.101+0000 7f6c6ffff700 1 -- 172.21.15.43:0/3793600149 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6c7804d2e0 con 0x7f6c8807b730 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.101+0000 7f6c867fc700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f6c70038fe0 0x7f6c7003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.101+0000 7f6c867fc700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f6c70038fe0 0x7f6c7003b4a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c006fd0 tx=0x7f6c7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.106+0000 7f6c6ffff700 1 -- 172.21.15.43:0/3793600149 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c78018350 con 0x7f6c8807b730 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.306+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f6c8819ef50 con 0x7f6c8807b730 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.306+0000 7f6c6ffff700 1 -- 172.21.15.43:0/3793600149 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f6c78023c20 con 0x7f6c8807b730 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f6c70038fe0 msgr2=0x7f6c7003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f6c70038fe0 0x7f6c7003b4a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c006fd0 tx=0x7f6c7c006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 msgr2=0x7f6c8819b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:22.872 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c8819b920 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6c78006af0 tx=0x7f6c78011a50 comp rx=0 tx=0).stop 2024-02-05T07:29:22.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 shutdown_connections 2024-02-05T07:29:22.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f6c70038fe0 0x7f6c7003b4a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:22.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 --2- 172.21.15.43:0/3793600149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c8807b730 0x7f6c8819b920 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:22.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 >> 172.21.15.43:0/3793600149 conn(0x7f6c88101070 msgr2=0x7f6c88101b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:22.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 shutdown_connections 2024-02-05T07:29:22.873 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:22.309+0000 7f6c8d513700 1 -- 172.21.15.43:0/3793600149 wait complete. 2024-02-05T07:29:24.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74bd504700 1 Processor -- start 2024-02-05T07:29:24.184 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74bd504700 1 -- start start 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74bd504700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74bd504700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8106480 con 0x7f74b8105a90 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74b6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74b6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33878/0 (socket says 172.21.15.43:33878) 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74b6ffd700 1 -- 172.21.15.43:0/853004711 learned_addr learned my addr 172.21.15.43:0/853004711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.323+0000 7f74b6ffd700 1 -- 172.21.15.43:0/853004711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b8106c90 con 0x7f74b8105a90 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74b6ffd700 1 --2- 172.21.15.43:0/853004711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8105eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f74a80098a0 tx=0x7f74a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79b387d60ca0ad6 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74b5ffb700 1 -- 172.21.15.43:0/853004711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a8004160 con 0x7f74b8105a90 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74b5ffb700 1 -- 172.21.15.43:0/853004711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f74a80042c0 con 0x7f74b8105a90 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74b5ffb700 1 -- 172.21.15.43:0/853004711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a8011850 con 0x7f74b8105a90 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74bd504700 1 -- 172.21.15.43:0/853004711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 msgr2=0x7f74b8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:24.185 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74bd504700 1 --2- 172.21.15.43:0/853004711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8105eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f74a80098a0 tx=0x7f74a8009bb0 comp rx=0 tx=0).stop 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74bd504700 1 -- 172.21.15.43:0/853004711 shutdown_connections 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74bd504700 1 --2- 172.21.15.43:0/853004711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8105eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74bd504700 1 -- 172.21.15.43:0/853004711 >> 172.21.15.43:0/853004711 conn(0x7f74b8101070 msgr2=0x7f74b8103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.324+0000 7f74bd504700 1 -- 172.21.15.43:0/853004711 shutdown_connections 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.325+0000 7f74bd504700 1 -- 172.21.15.43:0/853004711 wait complete. 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.325+0000 7f74bd504700 1 Processor -- start 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.325+0000 7f74bd504700 1 -- start start 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.325+0000 7f74bd504700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.325+0000 7f74bd504700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b8199c40 con 0x7f74b8105a90 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f74b6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f74b6ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33894/0 (socket says 172.21.15.43:33894) 2024-02-05T07:29:24.186 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f74b6ffd700 1 -- 172.21.15.43:0/2820722611 learned_addr learned my addr 172.21.15.43:0/2820722611 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f74b6ffd700 1 -- 172.21.15.43:0/2820722611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8009550 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f74b6ffd700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8199700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74a8006af0 tx=0x7f74a8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f749ffff700 1 -- 172.21.15.43:0/2820722611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a80036a0 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f749ffff700 1 -- 172.21.15.43:0/2820722611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f74a801a430 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b8199e40 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f749ffff700 1 -- 172.21.15.43:0/2820722611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f74a80193c0 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.326+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b819a2e0 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.327+0000 7f749ffff700 1 -- 172.21.15.43:0/2820722611 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f74a8019520 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.327+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b8063350 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.327+0000 7f749ffff700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f74a0038fe0 0x7f74a003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.327+0000 7f749ffff700 1 -- 172.21.15.43:0/2820722611 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f74a804d1e0 con 0x7f74b8105a90 2024-02-05T07:29:24.187 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.327+0000 7f74b67fc700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f74a0038fe0 0x7f74a003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.328+0000 7f74b67fc700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f74a0038fe0 0x7f74a003b4a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006fd0 tx=0x7f74ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.332+0000 7f749ffff700 1 -- 172.21.15.43:0/2820722611 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74a8018350 con 0x7f74b8105a90 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.647+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f74b819cbc0 con 0x7f74b8105a90 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.649+0000 7f749ffff700 1 -- 172.21.15.43:0/2820722611 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f74a8023c20 con 0x7f74b8105a90 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f74a0038fe0 msgr2=0x7f74a003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f74a0038fe0 0x7f74a003b4a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f74ac006fd0 tx=0x7f74ac006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 msgr2=0x7f74b8199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8199700 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f74a8006af0 tx=0x7f74a8011a50 comp rx=0 tx=0).stop 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 shutdown_connections 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f74a0038fe0 0x7f74a003b4a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 --2- 172.21.15.43:0/2820722611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74b8105a90 0x7f74b8199700 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:24.188 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 >> 172.21.15.43:0/2820722611 conn(0x7f74b8101070 msgr2=0x7f74b8101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 shutdown_connections 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-02-05T07:29:23.651+0000 7f74bd504700 1 -- 172.21.15.43:0/2820722611 wait complete. 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-05T07:29:24.189 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-02-05T07:29:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.190 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-02-05T07:29:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-05T07:29:24.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:24.190 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-02-05T07:29:24.228 INFO:tasks.cephadm:Fetching config... 2024-02-05T07:29:24.228 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:29:24.228 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-05T07:29:24.244 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-05T07:29:24.244 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:29:24.244 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-05T07:29:24.299 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-05T07:29:24.299 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:29:24.300 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-05T07:29:24.366 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-05T07:29:24.366 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:29:24.366 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-05T07:29:24.421 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-05T07:29:24.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXNT6KCVmV9DpONF9l0yPtMZE/PQFKKzd1WVvhGyrQq1F4kPWTOmxXBPBEfGqRcqImxXNjzi7aMj7PiPW5fyKxAH32zD+Ac/VqGgzDHKOJA+5Quawfcl1nLU4d2hB8wEUzN1Sc1spNadYszKQxcIrcqXh5hmg0aGXZ6dKBgx+B9CrNj6BlVOaOnn4wmgjOi5jIrmUnplCR+edBFSzCfYXD22GFAwkZjvf/3Y7J4Sj53ZJwlnjlfxZriH+lhoLGf8grTD4Y3Fy91CszWOYnE+Su4Hv+FUJYQxzwnFESSrqkWnTyNa6+c5h4h3yEee2PtRFvYBl+zMjIGpgOontx8PxTx8NtouSqkHy2Uzf2prXcc4Q3MDTbCVEQi2aEek3IMEaR8ktzetIopIwAgKC2cFgkrVTaWKYmTdfx22/HY1QdHAss/zffFMTAH/lYqmTSjq66ibralBUr50XLJJ+6tqvNp07xUfk0ZC0C0j7fzQDQzcGQ2hGlj2Ub3Tx7Bs0MdWc= ceph-f211dd04-c3f7-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-05T07:29:24.508 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXNT6KCVmV9DpONF9l0yPtMZE/PQFKKzd1WVvhGyrQq1F4kPWTOmxXBPBEfGqRcqImxXNjzi7aMj7PiPW5fyKxAH32zD+Ac/VqGgzDHKOJA+5Quawfcl1nLU4d2hB8wEUzN1Sc1spNadYszKQxcIrcqXh5hmg0aGXZ6dKBgx+B9CrNj6BlVOaOnn4wmgjOi5jIrmUnplCR+edBFSzCfYXD22GFAwkZjvf/3Y7J4Sj53ZJwlnjlfxZriH+lhoLGf8grTD4Y3Fy91CszWOYnE+Su4Hv+FUJYQxzwnFESSrqkWnTyNa6+c5h4h3yEee2PtRFvYBl+zMjIGpgOontx8PxTx8NtouSqkHy2Uzf2prXcc4Q3MDTbCVEQi2aEek3IMEaR8ktzetIopIwAgKC2cFgkrVTaWKYmTdfx22/HY1QdHAss/zffFMTAH/lYqmTSjq66ibralBUr50XLJJ+6tqvNp07xUfk0ZC0C0j7fzQDQzcGQ2hGlj2Ub3Tx7Bs0MdWc= ceph-f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:29:24.529 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXNT6KCVmV9DpONF9l0yPtMZE/PQFKKzd1WVvhGyrQq1F4kPWTOmxXBPBEfGqRcqImxXNjzi7aMj7PiPW5fyKxAH32zD+Ac/VqGgzDHKOJA+5Quawfcl1nLU4d2hB8wEUzN1Sc1spNadYszKQxcIrcqXh5hmg0aGXZ6dKBgx+B9CrNj6BlVOaOnn4wmgjOi5jIrmUnplCR+edBFSzCfYXD22GFAwkZjvf/3Y7J4Sj53ZJwlnjlfxZriH+lhoLGf8grTD4Y3Fy91CszWOYnE+Su4Hv+FUJYQxzwnFESSrqkWnTyNa6+c5h4h3yEee2PtRFvYBl+zMjIGpgOontx8PxTx8NtouSqkHy2Uzf2prXcc4Q3MDTbCVEQi2aEek3IMEaR8ktzetIopIwAgKC2cFgkrVTaWKYmTdfx22/HY1QdHAss/zffFMTAH/lYqmTSjq66ibralBUr50XLJJ+6tqvNp07xUfk0ZC0C0j7fzQDQzcGQ2hGlj2Ub3Tx7Bs0MdWc= ceph-f211dd04-c3f7-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-05T07:29:24.575 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDXNT6KCVmV9DpONF9l0yPtMZE/PQFKKzd1WVvhGyrQq1F4kPWTOmxXBPBEfGqRcqImxXNjzi7aMj7PiPW5fyKxAH32zD+Ac/VqGgzDHKOJA+5Quawfcl1nLU4d2hB8wEUzN1Sc1spNadYszKQxcIrcqXh5hmg0aGXZ6dKBgx+B9CrNj6BlVOaOnn4wmgjOi5jIrmUnplCR+edBFSzCfYXD22GFAwkZjvf/3Y7J4Sj53ZJwlnjlfxZriH+lhoLGf8grTD4Y3Fy91CszWOYnE+Su4Hv+FUJYQxzwnFESSrqkWnTyNa6+c5h4h3yEee2PtRFvYBl+zMjIGpgOontx8PxTx8NtouSqkHy2Uzf2prXcc4Q3MDTbCVEQi2aEek3IMEaR8ktzetIopIwAgKC2cFgkrVTaWKYmTdfx22/HY1QdHAss/zffFMTAH/lYqmTSjq66ibralBUr50XLJJ+6tqvNp07xUfk0ZC0C0j7fzQDQzcGQ2hGlj2Ub3Tx7Bs0MdWc= ceph-f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:29:24.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-05T07:29:24.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2820722611' entity='client.admin' 2024-02-05T07:29:24.830 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:29:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 -- 172.21.15.43:0/1049205855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 msgr2=0x7fa1a01045d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 --2- 172.21.15.43:0/1049205855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a01045d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa190009910 tx=0x7fa190009c20 comp rx=0 tx=0).stop 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 -- 172.21.15.43:0/1049205855 shutdown_connections 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 --2- 172.21.15.43:0/1049205855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a01045d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 -- 172.21.15.43:0/1049205855 >> 172.21.15.43:0/1049205855 conn(0x7fa1a00fbbf0 msgr2=0x7fa1a00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 -- 172.21.15.43:0/1049205855 shutdown_connections 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 -- 172.21.15.43:0/1049205855 wait complete. 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 Processor -- start 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.421+0000 7fa1a65f7700 1 -- start start 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa1a65f7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a0196180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa1a65f7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a01966c0 con 0x7fa1a01021e0 2024-02-05T07:29:27.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a0196180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19ffff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a0196180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33912/0 (socket says 172.21.15.43:33912) 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19ffff700 1 -- 172.21.15.43:0/3875202481 learned_addr learned my addr 172.21.15.43:0/3875202481 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19ffff700 1 -- 172.21.15.43:0/3875202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1900095f0 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19ffff700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a0196180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa190006010 tx=0x7fa190005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19d7fa700 1 -- 172.21.15.43:0/3875202481 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa19000b960 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19d7fa700 1 -- 172.21.15.43:0/3875202481 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa19000f460 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a01968c0 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.422+0000 7fa19d7fa700 1 -- 172.21.15.43:0/3875202481 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa190017440 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.423+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a0196d60 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.423+0000 7fa19d7fa700 1 -- 172.21.15.43:0/3875202481 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fa1900176b0 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.423+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a018ffd0 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.424+0000 7fa19d7fa700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa188038fe0 0x7fa18803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.424+0000 7fa19d7fa700 1 -- 172.21.15.43:0/3875202481 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa19004d160 con 0x7fa1a01021e0 2024-02-05T07:29:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.424+0000 7fa19f7fe700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa188038fe0 0x7fa18803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:27.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.424+0000 7fa19f7fe700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa188038fe0 0x7fa18803b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa194006fd0 tx=0x7fa194006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:27.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.429+0000 7fa19d7fa700 1 -- 172.21.15.43:0/3875202481 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa190015940 con 0x7fa1a01021e0 2024-02-05T07:29:27.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.630+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fa1a0106c80 con 0x7fa1a01021e0 2024-02-05T07:29:27.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.633+0000 7fa19d7fa700 1 -- 172.21.15.43:0/3875202481 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa190027030 con 0x7fa1a01021e0 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa188038fe0 msgr2=0x7fa18803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa188038fe0 0x7fa18803b4a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa194006fd0 tx=0x7fa194006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 msgr2=0x7fa1a0196180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a0196180 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa190006010 tx=0x7fa190005710 comp rx=0 tx=0).stop 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 shutdown_connections 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa188038fe0 0x7fa18803b4a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 --2- 172.21.15.43:0/3875202481 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1a01021e0 0x7fa1a0196180 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 >> 172.21.15.43:0/3875202481 conn(0x7fa1a00fbbf0 msgr2=0x7fa1a00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 shutdown_connections 2024-02-05T07:29:27.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:27.635+0000 7fa1a65f7700 1 -- 172.21.15.43:0/3875202481 wait complete. 2024-02-05T07:29:28.295 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-05T07:29:28.295 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-05T07:29:28.825 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:29:28.949 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3875202481' entity='client.admin' 2024-02-05T07:29:30.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:30.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:30.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:29:30.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:30.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:30.054 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.400+0000 7fcab546d700 1 -- 172.21.15.43:0/1477146330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 msgr2=0x7fcab0101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.400+0000 7fcab546d700 1 --2- 172.21.15.43:0/1477146330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab0101b70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0009910 tx=0x7fcaa0009c20 comp rx=0 tx=0).stop 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.400+0000 7fcab546d700 1 -- 172.21.15.43:0/1477146330 shutdown_connections 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.400+0000 7fcab546d700 1 --2- 172.21.15.43:0/1477146330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab0101b70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.400+0000 7fcab546d700 1 -- 172.21.15.43:0/1477146330 >> 172.21.15.43:0/1477146330 conn(0x7fcab00fccf0 msgr2=0x7fcab00ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.400+0000 7fcab546d700 1 -- 172.21.15.43:0/1477146330 shutdown_connections 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.400+0000 7fcab546d700 1 -- 172.21.15.43:0/1477146330 wait complete. 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcab546d700 1 Processor -- start 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcab546d700 1 -- start start 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcab546d700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab01961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcab546d700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab0196720 con 0x7fcab0101750 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcaaeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab01961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcaaeffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab01961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:33930/0 (socket says 172.21.15.43:33930) 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcaaeffd700 1 -- 172.21.15.43:0/3212884445 learned_addr learned my addr 172.21.15.43:0/3212884445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:30.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcaaeffd700 1 -- 172.21.15.43:0/3212884445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa00095f0 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.401+0000 7fcaaeffd700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab01961e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005660 tx=0x7fcaa0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.402+0000 7fca97fff700 1 -- 172.21.15.43:0/3212884445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcaa000ba90 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.402+0000 7fca97fff700 1 -- 172.21.15.43:0/3212884445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcaa000bbf0 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.402+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab0196920 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.402+0000 7fca97fff700 1 -- 172.21.15.43:0/3212884445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcaa000fea0 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.402+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab0196dc0 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.403+0000 7fca97fff700 1 -- 172.21.15.43:0/3212884445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fcaa002c440 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.403+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab018ffd0 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.403+0000 7fca97fff700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fca98038fe0 0x7fca9803b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.403+0000 7fca97fff700 1 -- 172.21.15.43:0/3212884445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcaa004d490 con 0x7fcab0101750 2024-02-05T07:29:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.403+0000 7fcaae7fc700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fca98038fe0 0x7fca9803b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.403+0000 7fcaae7fc700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fca98038fe0 0x7fca9803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4006fd0 tx=0x7fcaa4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.408+0000 7fca97fff700 1 -- 172.21.15.43:0/3212884445 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcaa0015940 con 0x7fcab0101750 2024-02-05T07:29:30.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.618+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fcab0106c80 con 0x7fca98038fe0 2024-02-05T07:29:30.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.621+0000 7fca97fff700 1 -- 172.21.15.43:0/3212884445 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fcab0106c80 con 0x7fca98038fe0 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fca98038fe0 msgr2=0x7fca9803b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fca98038fe0 0x7fca9803b4a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4006fd0 tx=0x7fcaa4006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 msgr2=0x7fcab01961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab01961e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcaa0005660 tx=0x7fcaa0005740 comp rx=0 tx=0).stop 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 shutdown_connections 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fca98038fe0 0x7fca9803b4a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 --2- 172.21.15.43:0/3212884445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcab0101750 0x7fcab01961e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 >> 172.21.15.43:0/3212884445 conn(0x7fcab00fccf0 msgr2=0x7fcab00fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 shutdown_connections 2024-02-05T07:29:30.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:30.623+0000 7fcab546d700 1 -- 172.21.15.43:0/3212884445 wait complete. 2024-02-05T07:29:31.361 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi050 2024-02-05T07:29:31.361 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:29:31.361 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.conf 2024-02-05T07:29:31.379 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:29:31.379 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:29:31.436 INFO:tasks.cephadm:Adding host smithi050 to orchestrator... 2024-02-05T07:29:31.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch host add smithi050 2024-02-05T07:29:31.678 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: Updating smithi043:/etc/ceph/ceph.conf 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:31.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:32 smithi043 ceph-mon[29879]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:29:32.811 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:32 smithi043 ceph-mon[29879]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:29:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.182+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/2282724119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 msgr2=0x7fd5d4107ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.182+0000 7fd5dc7fd700 1 --2- 172.21.15.43:0/2282724119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4107ae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009910 tx=0x7fd5d0009c20 comp rx=0 tx=0).stop 2024-02-05T07:29:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.182+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/2282724119 shutdown_connections 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.182+0000 7fd5dc7fd700 1 --2- 172.21.15.43:0/2282724119 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4107ae0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.182+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/2282724119 >> 172.21.15.43:0/2282724119 conn(0x7fd5d40fd230 msgr2=0x7fd5d40ff650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.182+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/2282724119 shutdown_connections 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.182+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/2282724119 wait complete. 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5dc7fd700 1 Processor -- start 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5dc7fd700 1 -- start start 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5dc7fd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5dc7fd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4196a70 con 0x7fd5d4107700 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5da599700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5da599700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57914/0 (socket says 172.21.15.43:57914) 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5da599700 1 -- 172.21.15.43:0/1422169472 learned_addr learned my addr 172.21.15.43:0/1422169472 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5da599700 1 -- 172.21.15.43:0/1422169472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00095f0 con 0x7fd5d4107700 2024-02-05T07:29:33.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.183+0000 7fd5da599700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4196530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0005660 tx=0x7fd5d0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.184+0000 7fd5c77fe700 1 -- 172.21.15.43:0/1422169472 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5d001d070 con 0x7fd5d4107700 2024-02-05T07:29:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.184+0000 7fd5c77fe700 1 -- 172.21.15.43:0/1422169472 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd5d000b6f0 con 0x7fd5d4107700 2024-02-05T07:29:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.184+0000 7fd5c77fe700 1 -- 172.21.15.43:0/1422169472 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd5d00037f0 con 0x7fd5d4107700 2024-02-05T07:29:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.184+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d4196cd0 con 0x7fd5d4107700 2024-02-05T07:29:33.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.184+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d4197150 con 0x7fd5d4107700 2024-02-05T07:29:33.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.184+0000 7fd5c77fe700 1 -- 172.21.15.43:0/1422169472 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fd5d0003950 con 0x7fd5d4107700 2024-02-05T07:29:33.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.185+0000 7fd5c77fe700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fd5c0038fe0 0x7fd5c003b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:33.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.185+0000 7fd5c77fe700 1 -- 172.21.15.43:0/1422169472 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd5d004cd10 con 0x7fd5d4107700 2024-02-05T07:29:33.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.185+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d4063350 con 0x7fd5d4107700 2024-02-05T07:29:33.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.185+0000 7fd5d9d98700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fd5c0038fe0 0x7fd5c003b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:33.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.185+0000 7fd5d9d98700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fd5c0038fe0 0x7fd5c003b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8006fd0 tx=0x7fd5c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:33.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.190+0000 7fd5c77fe700 1 -- 172.21.15.43:0/1422169472 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd5d0026080 con 0x7fd5d4107700 2024-02-05T07:29:33.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:33.397+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5d410afd0 con 0x7fd5c0038fe0 2024-02-05T07:29:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:34 smithi043 ceph-mon[29879]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:36.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:35 smithi043 ceph-mon[29879]: Deploying cephadm binary to smithi050 2024-02-05T07:29:37.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:36 smithi043 ceph-mon[29879]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:37.788 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi050' with addr '172.21.15.50' 2024-02-05T07:29:37.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.785+0000 7fd5c77fe700 1 -- 172.21.15.43:0/1422169472 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7fd5d410afd0 con 0x7fd5c0038fe0 2024-02-05T07:29:37.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fd5c0038fe0 msgr2=0x7fd5c003b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:37.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fd5c0038fe0 0x7fd5c003b4a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8006fd0 tx=0x7fd5c8006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:37.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 msgr2=0x7fd5d4196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:37.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4196530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0005660 tx=0x7fd5d0005740 comp rx=0 tx=0).stop 2024-02-05T07:29:37.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 shutdown_connections 2024-02-05T07:29:37.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fd5c0038fe0 0x7fd5c003b4a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:37.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 --2- 172.21.15.43:0/1422169472 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5d4107700 0x7fd5d4196530 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:37.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 >> 172.21.15.43:0/1422169472 conn(0x7fd5d40fd230 msgr2=0x7fd5d4109620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:37.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 shutdown_connections 2024-02-05T07:29:37.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:37.788+0000 7fd5dc7fd700 1 -- 172.21.15.43:0/1422169472 wait complete. 2024-02-05T07:29:38.399 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-05T07:29:38.644 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:29:39.003 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:38 smithi043 ceph-mon[29879]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:39.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:39.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:38 smithi043 ceph-mon[29879]: Added host smithi050 2024-02-05T07:29:39.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:39.004 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:39.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.980+0000 7ff018544700 1 -- 172.21.15.43:0/3097850456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 msgr2=0x7ff010107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:39.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.980+0000 7ff018544700 1 --2- 172.21.15.43:0/3097850456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff010107b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff00000b0a0 tx=0x7ff00000b3b0 comp rx=0 tx=0).stop 2024-02-05T07:29:39.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.980+0000 7ff018544700 1 -- 172.21.15.43:0/3097850456 shutdown_connections 2024-02-05T07:29:39.982 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:39 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.980+0000 7ff018544700 1 --2- 172.21.15.43:0/3097850456 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff010107b90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.980+0000 7ff018544700 1 -- 172.21.15.43:0/3097850456 >> 172.21.15.43:0/3097850456 conn(0x7ff0100fb8a0 msgr2=0x7ff0100fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.980+0000 7ff018544700 1 -- 172.21.15.43:0/3097850456 shutdown_connections 2024-02-05T07:29:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.980+0000 7ff018544700 1 -- 172.21.15.43:0/3097850456 wait complete. 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff018544700 1 Processor -- start 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff018544700 1 -- start start 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff018544700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff0101986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff018544700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010198c20 con 0x7ff0101077b0 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff0162e0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff0101986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff0162e0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff0101986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57936/0 (socket says 172.21.15.43:57936) 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff0162e0700 1 -- 172.21.15.43:0/2972715816 learned_addr learned my addr 172.21.15.43:0/2972715816 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff0162e0700 1 -- 172.21.15.43:0/2972715816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff000009d00 con 0x7ff0101077b0 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.981+0000 7ff0162e0700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff0101986e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff000009610 tx=0x7ff00000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.982+0000 7ff00f7fe700 1 -- 172.21.15.43:0/2972715816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff00000e040 con 0x7ff0101077b0 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.982+0000 7ff00f7fe700 1 -- 172.21.15.43:0/2972715816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff000003c10 con 0x7ff0101077b0 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.982+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff010198e20 con 0x7ff0101077b0 2024-02-05T07:29:39.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.982+0000 7ff00f7fe700 1 -- 172.21.15.43:0/2972715816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff00001d470 con 0x7ff0101077b0 2024-02-05T07:29:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.982+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff010199320 con 0x7ff0101077b0 2024-02-05T07:29:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.982+0000 7ff00f7fe700 1 -- 172.21.15.43:0/2972715816 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff000019070 con 0x7ff0101077b0 2024-02-05T07:29:39.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.983+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff010063350 con 0x7ff0101077b0 2024-02-05T07:29:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.983+0000 7ff00f7fe700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7feff8038f80 0x7feff803b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.983+0000 7ff00f7fe700 1 -- 172.21.15.43:0/2972715816 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff00004d010 con 0x7ff0101077b0 2024-02-05T07:29:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.983+0000 7ff015adf700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7feff8038f80 0x7feff803b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.983+0000 7ff015adf700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7feff8038f80 0x7feff803b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff004006fd0 tx=0x7ff004006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:39.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:39.988+0000 7ff00f7fe700 1 -- 172.21.15.43:0/2972715816 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff000004040 con 0x7ff0101077b0 2024-02-05T07:29:40.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.197+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff01010afd0 con 0x7feff8038f80 2024-02-05T07:29:40.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.198+0000 7ff00f7fe700 1 -- 172.21.15.43:0/2972715816 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7ff01010afd0 con 0x7feff8038f80 2024-02-05T07:29:40.199 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:29:40.199 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}] 2024-02-05T07:29:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7feff8038f80 msgr2=0x7feff803b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7feff8038f80 0x7feff803b440 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff004006fd0 tx=0x7ff004006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 msgr2=0x7ff0101986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff0101986e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff000009610 tx=0x7ff00000bd40 comp rx=0 tx=0).stop 2024-02-05T07:29:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 shutdown_connections 2024-02-05T07:29:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7feff8038f80 0x7feff803b440 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 --2- 172.21.15.43:0/2972715816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0101077b0 0x7ff0101986e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 >> 172.21.15.43:0/2972715816 conn(0x7ff0100fb8a0 msgr2=0x7ff0100fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 shutdown_connections 2024-02-05T07:29:40.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:40.200+0000 7ff018544700 1 -- 172.21.15.43:0/2972715816 wait complete. 2024-02-05T07:29:40.765 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-05T07:29:40.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd crush tunables default 2024-02-05T07:29:41.004 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:29:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:40 smithi043 ceph-mon[29879]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:42.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:41 smithi043 ceph-mon[29879]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:29:42.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.494+0000 7f36459b5700 1 -- 172.21.15.43:0/77918534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 msgr2=0x7f3640106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:42.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.494+0000 7f36459b5700 1 --2- 172.21.15.43:0/77918534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f3640106a50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f363800b0a0 tx=0x7f363800b3b0 comp rx=0 tx=0).stop 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.494+0000 7f36459b5700 1 -- 172.21.15.43:0/77918534 shutdown_connections 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.494+0000 7f36459b5700 1 --2- 172.21.15.43:0/77918534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f3640106a50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.494+0000 7f36459b5700 1 -- 172.21.15.43:0/77918534 >> 172.21.15.43:0/77918534 conn(0x7f36400fc0d0 msgr2=0x7f36400fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.494+0000 7f36459b5700 1 -- 172.21.15.43:0/77918534 shutdown_connections 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.494+0000 7f36459b5700 1 -- 172.21.15.43:0/77918534 wait complete. 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f36459b5700 1 Processor -- start 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f36459b5700 1 -- start start 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f36459b5700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f36401965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f36459b5700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3640196b10 con 0x7f3640106670 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f363effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f36401965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:42.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f363effd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f36401965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39568/0 (socket says 172.21.15.43:39568) 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f363effd700 1 -- 172.21.15.43:0/927673057 learned_addr learned my addr 172.21.15.43:0/927673057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f363effd700 1 -- 172.21.15.43:0/927673057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009d00 con 0x7f3640106670 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.495+0000 7f363effd700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f36401965d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f363800b830 tx=0x7f363800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.496+0000 7f36449b3700 1 -- 172.21.15.43:0/927673057 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f363800e040 con 0x7f3640106670 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.496+0000 7f36449b3700 1 -- 172.21.15.43:0/927673057 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3638003c10 con 0x7f3640106670 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.496+0000 7f36449b3700 1 -- 172.21.15.43:0/927673057 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f363801d470 con 0x7f3640106670 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.496+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3640196d10 con 0x7f3640106670 2024-02-05T07:29:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.496+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3640197210 con 0x7f3640106670 2024-02-05T07:29:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.496+0000 7f36449b3700 1 -- 172.21.15.43:0/927673057 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f3638019070 con 0x7f3640106670 2024-02-05T07:29:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.496+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3640063350 con 0x7f3640106670 2024-02-05T07:29:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.497+0000 7f36449b3700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f362c038fd0 0x7f362c03b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:42.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.497+0000 7f36449b3700 1 -- 172.21.15.43:0/927673057 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f363804d1c0 con 0x7f3640106670 2024-02-05T07:29:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.497+0000 7f363e7fc700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f362c038fd0 0x7f362c03b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:42.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.497+0000 7f363e7fc700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f362c038fd0 0x7f362c03b490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3628006fd0 tx=0x7f3628006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:42.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.502+0000 7f36449b3700 1 -- 172.21.15.43:0/927673057 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3638026780 con 0x7f3640106670 2024-02-05T07:29:42.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.710+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f36401978e0 con 0x7f3640106670 2024-02-05T07:29:42.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.876+0000 7f36449b3700 1 -- 172.21.15.43:0/927673057 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f3638017030 con 0x7f3640106670 2024-02-05T07:29:42.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f362c038fd0 msgr2=0x7f362c03b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f362c038fd0 0x7f362c03b490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3628006fd0 tx=0x7f3628006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 msgr2=0x7f36401965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f36401965d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f363800b830 tx=0x7f363800bd40 comp rx=0 tx=0).stop 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 shutdown_connections 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f362c038fd0 0x7f362c03b490 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 --2- 172.21.15.43:0/927673057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3640106670 0x7f36401965d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 >> 172.21.15.43:0/927673057 conn(0x7f36400fc0d0 msgr2=0x7f364006a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 shutdown_connections 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:29:42.879+0000 7f36459b5700 1 -- 172.21.15.43:0/927673057 wait complete. 2024-02-05T07:29:42.881 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-02-05T07:29:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:42 smithi043 ceph-mon[29879]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/927673057' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T07:29:43.370 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-02-05T07:29:43.371 INFO:tasks.cephadm:Adding mon.c on smithi043 2024-02-05T07:29:43.371 INFO:tasks.cephadm:Adding mon.b on smithi050 2024-02-05T07:29:43.371 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply mon '3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b' 2024-02-05T07:29:43.682 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-05T07:29:43.730 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-05T07:29:44.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/927673057' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T07:29:44.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:29:44.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:44.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:44.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:44.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:44.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:29:44.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:44.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:29:45.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:44 smithi043 ceph-mon[29879]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:45.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:44 smithi043 ceph-mon[29879]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:29:45.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:44 smithi043 ceph-mon[29879]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:29:45.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:45.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:45.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:45.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.428+0000 7f60e870c700 1 -- 172.21.15.50:0/2849008220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 msgr2=0x7f60e0106a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.428+0000 7f60e870c700 1 --2- 172.21.15.50:0/2849008220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0106a70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009910 tx=0x7f60dc009c20 comp rx=0 tx=0).stop 2024-02-05T07:29:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.428+0000 7f60e870c700 1 -- 172.21.15.50:0/2849008220 shutdown_connections 2024-02-05T07:29:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.428+0000 7f60e870c700 1 --2- 172.21.15.50:0/2849008220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0106a70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.428+0000 7f60e870c700 1 -- 172.21.15.50:0/2849008220 >> 172.21.15.50:0/2849008220 conn(0x7f60e00fc110 msgr2=0x7f60e00fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.428+0000 7f60e870c700 1 -- 172.21.15.50:0/2849008220 shutdown_connections 2024-02-05T07:29:45.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.428+0000 7f60e870c700 1 -- 172.21.15.50:0/2849008220 wait complete. 2024-02-05T07:29:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.429+0000 7f60e870c700 1 Processor -- start 2024-02-05T07:29:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.429+0000 7f60e870c700 1 -- start start 2024-02-05T07:29:45.431 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.429+0000 7f60e870c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.429+0000 7f60e870c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60e006ab50 con 0x7f60e0106690 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.429+0000 7f60e64a8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60e64a8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0197450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.50:36996/0 (socket says 172.21.15.50:36996) 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60e64a8700 1 -- 172.21.15.50:0/3121034386 learned_addr learned my addr 172.21.15.50:0/3121034386 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60e64a8700 1 -- 172.21.15.50:0/3121034386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc0095f0 con 0x7f60e0106690 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60e64a8700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0197450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005660 tx=0x7f60dc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60d77fe700 1 -- 172.21.15.50:0/3121034386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60dc01d070 con 0x7f60e0106690 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60e0197a50 con 0x7f60e0106690 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60d77fe700 1 -- 172.21.15.50:0/3121034386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f60dc00b750 con 0x7f60e0106690 2024-02-05T07:29:45.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60d77fe700 1 -- 172.21.15.50:0/3121034386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60dc003680 con 0x7f60e0106690 2024-02-05T07:29:45.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.430+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60e0197ed0 con 0x7f60e0106690 2024-02-05T07:29:45.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.431+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60e0105980 con 0x7f60e0106690 2024-02-05T07:29:45.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.431+0000 7f60d77fe700 1 -- 172.21.15.50:0/3121034386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f60dc004260 con 0x7f60e0106690 2024-02-05T07:29:45.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.431+0000 7f60d77fe700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f60cc038f80 0x7f60cc03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:45.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.431+0000 7f60d77fe700 1 -- 172.21.15.50:0/3121034386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f60dc04ce40 con 0x7f60e0106690 2024-02-05T07:29:45.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.432+0000 7f60e5ca7700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f60cc038f80 0x7f60cc03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:45.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.432+0000 7f60e5ca7700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f60cc038f80 0x7f60cc03b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006fd0 tx=0x7f60d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:45.437 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.437+0000 7f60d77fe700 1 -- 172.21.15.50:0/3121034386 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f60dc029750 con 0x7f60e0106690 2024-02-05T07:29:45.653 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.652+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f60e0198120 con 0x7f60cc038f80 2024-02-05T07:29:45.660 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.658+0000 7f60d77fe700 1 -- 172.21.15.50:0/3121034386 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f60e0198120 con 0x7f60cc038f80 2024-02-05T07:29:45.660 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled mon update... 2024-02-05T07:29:45.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.660+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f60cc038f80 msgr2=0x7f60cc03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:45.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.660+0000 7f60e870c700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f60cc038f80 0x7f60cc03b440 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f60d0006fd0 tx=0x7f60d0006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:45.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.660+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 msgr2=0x7f60e0197450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:45.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.660+0000 7f60e870c700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0197450 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f60dc005660 tx=0x7f60dc005740 comp rx=0 tx=0).stop 2024-02-05T07:29:45.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.660+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 shutdown_connections 2024-02-05T07:29:45.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.660+0000 7f60e870c700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f60cc038f80 0x7f60cc03b440 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:45.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.660+0000 7f60e870c700 1 --2- 172.21.15.50:0/3121034386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60e0106690 0x7f60e0197450 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:45.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.661+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 >> 172.21.15.50:0/3121034386 conn(0x7f60e00fc110 msgr2=0x7f60e00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:45.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.661+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 shutdown_connections 2024-02-05T07:29:45.663 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:45.661+0000 7f60e870c700 1 -- 172.21.15.50:0/3121034386 wait complete. 2024-02-05T07:29:46.219 DEBUG:teuthology.orchestra.run.smithi043:mon.c> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.c.service 2024-02-05T07:29:46.222 DEBUG:teuthology.orchestra.run.smithi050:mon.b> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.b.service 2024-02-05T07:29:46.224 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T07:29:46.224 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:46.250 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:29:46.251 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:46.933 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:29:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:46 smithi043 ceph-mon[29879]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:46 smithi043 ceph-mon[29879]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:47.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:46 smithi043 ceph-mon[29879]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b;count:3 2024-02-05T07:29:47.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:46 smithi043 ceph-mon[29879]: Deploying daemon mon.b on smithi050 2024-02-05T07:29:49.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:48 smithi043 ceph-mon[29879]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:50.033 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.029+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2142306968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 msgr2=0x7f5b14107ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.029+0000 7f5b1a1f1700 1 --2- 172.21.15.50:0/2142306968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14107ad0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b04009910 tx=0x7f5b04009c20 comp rx=0 tx=0).stop 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.029+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2142306968 shutdown_connections 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.029+0000 7f5b1a1f1700 1 --2- 172.21.15.50:0/2142306968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14107ad0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.029+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2142306968 >> 172.21.15.50:0/2142306968 conn(0x7f5b140fd200 msgr2=0x7f5b140ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.029+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2142306968 shutdown_connections 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.029+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2142306968 wait complete. 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.030+0000 7f5b1a1f1700 1 Processor -- start 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.030+0000 7f5b1a1f1700 1 -- start start 2024-02-05T07:29:50.034 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.030+0000 7f5b1a1f1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.030+0000 7f5b1a1f1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b14196a70 con 0x7f5b141076f0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.030+0000 7f5b137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.030+0000 7f5b137fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.50:37026/0 (socket says 172.21.15.50:37026) 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.030+0000 7f5b137fe700 1 -- 172.21.15.50:0/2759718565 learned_addr learned my addr 172.21.15.50:0/2759718565 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.031+0000 7f5b137fe700 1 -- 172.21.15.50:0/2759718565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b040095f0 con 0x7f5b141076f0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.031+0000 7f5b137fe700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14196530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b04006010 tx=0x7f5b04005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.031+0000 7f5b10ff9700 1 -- 172.21.15.50:0/2759718565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b0401d070 con 0x7f5b141076f0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.031+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b14196cd0 con 0x7f5b141076f0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.031+0000 7f5b10ff9700 1 -- 172.21.15.50:0/2759718565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5b0400b6f0 con 0x7f5b141076f0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.031+0000 7f5b10ff9700 1 -- 172.21.15.50:0/2759718565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5b04003810 con 0x7f5b141076f0 2024-02-05T07:29:50.035 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.031+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b14197150 con 0x7f5b141076f0 2024-02-05T07:29:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.032+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b14063350 con 0x7f5b141076f0 2024-02-05T07:29:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.032+0000 7f5b10ff9700 1 -- 172.21.15.50:0/2759718565 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f5b04004390 con 0x7f5b141076f0 2024-02-05T07:29:50.036 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.032+0000 7f5b10ff9700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f5afc038f80 0x7f5afc03b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.032+0000 7f5b10ff9700 1 -- 172.21.15.50:0/2759718565 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5b0404cda0 con 0x7f5b141076f0 2024-02-05T07:29:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.032+0000 7f5b12ffd700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f5afc038f80 0x7f5afc03b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:50.037 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.033+0000 7f5b12ffd700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f5afc038f80 0x7f5afc03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5b08006fd0 tx=0x7f5b08006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:50.040 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.038+0000 7f5b10ff9700 1 -- 172.21.15.50:0/2759718565 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b0400bb40 con 0x7f5b141076f0 2024-02-05T07:29:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.339+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5b14197890 con 0x7f5b141076f0 2024-02-05T07:29:50.341 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.340+0000 7f5b10ff9700 1 -- 172.21.15.50:0/2759718565 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+745 (secure 0 0 0) 0x7f5b04029360 con 0x7f5b141076f0 2024-02-05T07:29:50.342 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:29:50.342 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":1,"fsid":"f211dd04-c3f7-11ee-95b6-87774f69a715","modified":"2024-02-05T07:28:11.879318Z","created":"2024-02-05T07:28:11.879318Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f5afc038f80 msgr2=0x7f5afc03b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f5afc038f80 0x7f5afc03b440 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5b08006fd0 tx=0x7f5b08006e40 comp rx=0 tx=0).stop 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 msgr2=0x7f5b14196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14196530 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5b04006010 tx=0x7f5b04005710 comp rx=0 tx=0).stop 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 shutdown_connections 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f5afc038f80 0x7f5afc03b440 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 --2- 172.21.15.50:0/2759718565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b141076f0 0x7f5b14196530 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 >> 172.21.15.50:0/2759718565 conn(0x7f5b140fd200 msgr2=0x7f5b14109630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 shutdown_connections 2024-02-05T07:29:50.344 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:50.342+0000 7f5b1a1f1700 1 -- 172.21.15.50:0/2759718565 wait complete. 2024-02-05T07:29:50.345 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 1 2024-02-05T07:29:51.171 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 podman[33454]: 2024-02-05 07:29:51.170141608 +0000 UTC m=+0.118888443 container create 18b270fd5e95df227e43d5f6ca9d9359caa5d46c2a1f9ce92b1df4d24dee26b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-02-05T07:29:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:50 smithi043 ceph-mon[29879]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/2759718565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 podman[33454]: 2024-02-05 07:29:51.356248556 +0000 UTC m=+0.304995399 container init 18b270fd5e95df227e43d5f6ca9d9359caa5d46c2a1f9ce92b1df4d24dee26b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b, org.label-schema.build-date=20240131, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 podman[33454]: 2024-02-05 07:29:51.361564583 +0000 UTC m=+0.310311422 container start 18b270fd5e95df227e43d5f6ca9d9359caa5d46c2a1f9ce92b1df4d24dee26b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: pidfile_write: ignore empty --pid-file 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: load: jerasure load: lrc 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: RocksDB version: 7.9.2 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Git sha 0 2024-02-05T07:29:51.449 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: DB SUMMARY 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: DB Session ID: R62MSYQR83MKSJVES62F 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: CURRENT file: CURRENT 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.error_if_exists: 0 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.create_if_missing: 0 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.paranoid_checks: 1 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T07:29:51.450 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T07:29:51.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T07:29:51.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.env: 0x55bd0dbe2ae0 2024-02-05T07:29:51.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T07:29:51.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.info_log: 0x55bd0f43d8c0 2024-02-05T07:29:51.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.statistics: (nil) 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.use_fsync: 0 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_log_file_size: 0 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.allow_fallocate: 1 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T07:29:51.452 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.use_direct_reads: 0 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.db_log_dir: 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.wal_dir: 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.write_buffer_manager: 0x55bd0f411400 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T07:29:51.453 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.unordered_write: 0 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.row_cache: None 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.wal_filter: None 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T07:29:51.454 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.two_write_queues: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.wal_compression: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.atomic_flush: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.log_readahead_size: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T07:29:51.455 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_background_jobs: 2 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_background_compactions: -1 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_subcompactions: 1 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_open_files: -1 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T07:29:51.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_background_flushes: -1 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Compression algorithms supported: 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kZSTD supported: 0 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kXpressCompression supported: 0 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kZlibCompression supported: 1 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kSnappyCompression supported: 1 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kLZ4Compression supported: 1 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: kBZip2Compression supported: 0 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T07:29:51.457 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.merge_operator: 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_filter: None 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bd0f43d9c0) 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout: cache_index_and_filter_blocks: 1 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout: pin_top_level_index_and_filter: 1 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_type: 0 2024-02-05T07:29:51.458 INFO:journalctl@ceph.mon.b.smithi050.stdout: data_block_index_type: 0 2024-02-05T07:29:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_shortening: 1 2024-02-05T07:29:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T07:29:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout: checksum: 4 2024-02-05T07:29:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout: no_block_cache: 0 2024-02-05T07:29:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache: 0x55bd0f441090 2024-02-05T07:29:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_name: BinnedLRUCache 2024-02-05T07:29:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_options: 2024-02-05T07:29:51.460 INFO:journalctl@ceph.mon.b.smithi050.stdout: capacity : 536870912 2024-02-05T07:29:51.460 INFO:journalctl@ceph.mon.b.smithi050.stdout: num_shard_bits : 4 2024-02-05T07:29:51.460 INFO:journalctl@ceph.mon.b.smithi050.stdout: strict_capacity_limit : 0 2024-02-05T07:29:51.460 INFO:journalctl@ceph.mon.b.smithi050.stdout: high_pri_pool_ratio: 0.000 2024-02-05T07:29:51.460 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_cache_compressed: (nil) 2024-02-05T07:29:51.460 INFO:journalctl@ceph.mon.b.smithi050.stdout: persistent_cache: (nil) 2024-02-05T07:29:51.461 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_size: 4096 2024-02-05T07:29:51.461 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_size_deviation: 10 2024-02-05T07:29:51.461 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_restart_interval: 16 2024-02-05T07:29:51.461 INFO:journalctl@ceph.mon.b.smithi050.stdout: index_block_restart_interval: 1 2024-02-05T07:29:51.461 INFO:journalctl@ceph.mon.b.smithi050.stdout: metadata_block_size: 4096 2024-02-05T07:29:51.461 INFO:journalctl@ceph.mon.b.smithi050.stdout: partition_filters: 0 2024-02-05T07:29:51.462 INFO:journalctl@ceph.mon.b.smithi050.stdout: use_delta_encoding: 1 2024-02-05T07:29:51.462 INFO:journalctl@ceph.mon.b.smithi050.stdout: filter_policy: bloomfilter 2024-02-05T07:29:51.462 INFO:journalctl@ceph.mon.b.smithi050.stdout: whole_key_filtering: 1 2024-02-05T07:29:51.462 INFO:journalctl@ceph.mon.b.smithi050.stdout: verify_compression: 0 2024-02-05T07:29:51.462 INFO:journalctl@ceph.mon.b.smithi050.stdout: read_amp_bytes_per_bit: 0 2024-02-05T07:29:51.462 INFO:journalctl@ceph.mon.b.smithi050.stdout: format_version: 5 2024-02-05T07:29:51.463 INFO:journalctl@ceph.mon.b.smithi050.stdout: enable_index_compression: 1 2024-02-05T07:29:51.463 INFO:journalctl@ceph.mon.b.smithi050.stdout: block_align: 0 2024-02-05T07:29:51.463 INFO:journalctl@ceph.mon.b.smithi050.stdout: max_auto_readahead_size: 262144 2024-02-05T07:29:51.463 INFO:journalctl@ceph.mon.b.smithi050.stdout: prepopulate_block_cache: 0 2024-02-05T07:29:51.463 INFO:journalctl@ceph.mon.b.smithi050.stdout: initial_auto_readahead_size: 8192 2024-02-05T07:29:51.463 INFO:journalctl@ceph.mon.b.smithi050.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T07:29:51.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T07:29:51.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T07:29:51.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression: NoCompression 2024-02-05T07:29:51.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T07:29:51.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T07:29:51.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T07:29:51.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.num_levels: 7 2024-02-05T07:29:51.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T07:29:51.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T07:29:51.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T07:29:51.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T07:29:51.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T07:29:51.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T07:29:51.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T07:29:51.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:29:51.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T07:29:51.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T07:29:51.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:29:51.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:29:51.467 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T07:29:51.467 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T07:29:51.467 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T07:29:51.467 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T07:29:51.467 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:29:51.468 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:29:51.468 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T07:29:51.468 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T07:29:51.468 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:29:51.468 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T07:29:51.468 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T07:29:51.468 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T07:29:51.469 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T07:29:51.469 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T07:29:51.469 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T07:29:51.469 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T07:29:51.469 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T07:29:51.469 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T07:29:51.470 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T07:29:51.471 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T07:29:51.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T07:29:51.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.table_properties_collectors: 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.inplace_update_support: 0 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T07:29:51.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.bloom_locality: 0 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.max_successive_merges: 0 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.ttl: 2592000 2024-02-05T07:29:51.475 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.enable_blob_files: false 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.min_blob_size: 0 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T07:29:51.476 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 42bad8d3-b178-43e7-87df-b7714317efde 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118191392048, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118191392469, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707118191, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "42bad8d3-b178-43e7-87df-b7714317efde", "db_session_id": "R62MSYQR83MKSJVES62F", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118191392563, "job": 1, "event": "recovery_finished"} 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T07:29:51.477 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bd0f532000 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: DB pointer 0x55bd0f51c000 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** DB Stats ** 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-05T07:29:51.478 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** Compaction Stats [default] ** 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T07:29:51.479 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache BinnedLRUCache@0x55bd0f441090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-05T07:29:51.480 INFO:journalctl@ceph.mon.b.smithi050.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: using public_addr v2:172.21.15.50:0/0 -> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: starting mon.b rank -1 at public addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] at bind addrs [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(???) e0 preinit fsid f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).mds e1 new map 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout: e1 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout: legacy client fscid: -1 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout: 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout: No filesystems configured 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-05T07:29:51.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/927673057' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T07:29:51.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/927673057' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.483 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:29:51.484 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b;count:3 2024-02-05T07:29:51.485 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: Deploying daemon mon.b on smithi050 2024-02-05T07:29:51.485 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:51.485 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:51.485 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/2759718565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:29:51.485 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 ceph-mon[33492]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-05T07:29:51.485 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 bash[33454]: 18b270fd5e95df227e43d5f6ca9d9359caa5d46c2a1f9ce92b1df4d24dee26b7 2024-02-05T07:29:51.485 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:51 smithi050 systemd[1]: Started Ceph mon.b for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:29:51.949 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T07:29:51.950 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T07:29:52.193 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:29:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: mon.a calling monitor election 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: mon.b calling monitor election 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: fsmap 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:29:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: mgrmap e13: y(active, since 41s) 2024-02-05T07:29:56.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: overall HEALTH_OK 2024-02-05T07:29:56.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:56.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:56.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:56.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:29:56.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:56.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:56.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:29:56.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: mon.a calling monitor election 2024-02-05T07:29:56.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:56.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: mon.b calling monitor election 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: fsmap 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: mgrmap e13: y(active, since 41s) 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: overall HEALTH_OK 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:29:56.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:29:56.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.847+0000 7f436fb6a700 1 -- 172.21.15.50:0/1381253970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43681066d0 msgr2=0x7f4354005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.847+0000 7f436fb6a700 1 --2- 172.21.15.50:0/1381253970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43681066d0 0x7f4354005610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4358005fa0 tx=0x7f435800ffc0 comp rx=0 tx=0).stop 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.847+0000 7f436fb6a700 1 -- 172.21.15.50:0/1381253970 shutdown_connections 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.847+0000 7f436fb6a700 1 --2- 172.21.15.50:0/1381253970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43681066d0 0x7f4354005610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.847+0000 7f436fb6a700 1 -- 172.21.15.50:0/1381253970 >> 172.21.15.50:0/1381253970 conn(0x7f43680fbe30 msgr2=0x7f43680fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.847+0000 7f436fb6a700 1 -- 172.21.15.50:0/1381253970 shutdown_connections 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.847+0000 7f436fb6a700 1 -- 172.21.15.50:0/1381253970 wait complete. 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436fb6a700 1 Processor -- start 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436fb6a700 1 -- start start 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436fb6a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 0x7f4368197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:56.851 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436fb6a700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368197a80 0x7f4368197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436fb6a700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4368100db0 con 0x7f4368197a80 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436fb6a700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43681a05b0 con 0x7f43681066d0 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436d906700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 0x7f4368197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436d906700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 0x7f4368197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:47352/0 (socket says 172.21.15.50:47352) 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436d906700 1 -- 172.21.15.50:0/3127904970 learned_addr learned my addr 172.21.15.50:0/3127904970 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.848+0000 7f436d105700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368197a80 0x7f4368197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d906700 1 -- 172.21.15.50:0/3127904970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 msgr2=0x7f4368197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d906700 1 -- 172.21.15.50:0/3127904970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 msgr2=0x7f4368197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d906700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 0x7f4368197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d906700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 0x7f4368197540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d105700 1 -- 172.21.15.50:0/3127904970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 msgr2=0x7f4368197540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d105700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 0x7f4368197540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d105700 1 -- 172.21.15.50:0/3127904970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435800f9c0 con 0x7f4368197a80 2024-02-05T07:29:56.852 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436d105700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368197a80 0x7f4368197f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f436400b4d0 tx=0x7f436400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f435effd700 1 -- 172.21.15.50:0/3127904970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4364011630 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f435effd700 1 -- 172.21.15.50:0/3127904970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4364011c10 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f435effd700 1 -- 172.21.15.50:0/3127904970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f4364010b00 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43681a0840 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.849+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43681a0d90 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.850+0000 7f435effd700 1 -- 172.21.15.50:0/3127904970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f4364010c60 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.850+0000 7f435effd700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f434c038d90 0x7f434c03b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.850+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368063350 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.850+0000 7f435effd700 1 -- 172.21.15.50:0/3127904970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f436404b0e0 con 0x7f4368197a80 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.851+0000 7f436d906700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f434c038d90 0x7f434c03b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:29:56.854 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.851+0000 7f436d906700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f434c038d90 0x7f434c03b250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f435800f470 tx=0x7f435800f3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:29:56.858 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:56.856+0000 7f435effd700 1 -- 172.21.15.50:0/3127904970 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f436401a410 con 0x7f4368197a80 2024-02-05T07:29:57.144 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.142+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f43681003f0 con 0x7f4368197a80 2024-02-05T07:29:57.145 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.143+0000 7f435effd700 1 -- 172.21.15.50:0/3127904970 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1011 (secure 0 0 0) 0x7f4364018310 con 0x7f4368197a80 2024-02-05T07:29:57.145 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:29:57.145 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":2,"fsid":"f211dd04-c3f7-11ee-95b6-87774f69a715","modified":"2024-02-05T07:29:51.413894Z","created":"2024-02-05T07:28:11.879318Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-05T07:29:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f434c038d90 msgr2=0x7f434c03b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f434c038d90 0x7f434c03b250 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f435800f470 tx=0x7f435800f3c0 comp rx=0 tx=0).stop 2024-02-05T07:29:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368197a80 msgr2=0x7f4368197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:29:57.147 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368197a80 0x7f4368197f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f436400b4d0 tx=0x7f436400b890 comp rx=0 tx=0).stop 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 shutdown_connections 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f434c038d90 0x7f434c03b250 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43681066d0 0x7f4368197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 --2- 172.21.15.50:0/3127904970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4368197a80 0x7f4368197f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 >> 172.21.15.50:0/3127904970 conn(0x7f43680fbe30 msgr2=0x7f43680fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 shutdown_connections 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:29:57.145+0000 7f436fb6a700 1 -- 172.21.15.50:0/3127904970 wait complete. 2024-02-05T07:29:57.148 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 2 2024-02-05T07:29:57.703 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:57 smithi050 ceph-mon[33492]: Deploying daemon mon.c on smithi043 2024-02-05T07:29:57.703 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/3127904970' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:29:57.703 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:57 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:57.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:57 smithi043 ceph-mon[29879]: Deploying daemon mon.c on smithi043 2024-02-05T07:29:57.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/3127904970' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:29:57.781 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:57 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:29:58.704 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-05T07:29:58.705 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-05T07:29:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:29:58 smithi043 ceph-mon[29879]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:58.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:29:58 smithi050 ceph-mon[33492]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:29:58.957 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:29:59.723 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 systemd[1]: Starting Ceph mon.c for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 podman[41406]: 2024-02-05 07:29:59.625218383 +0000 UTC m=+0.017633753 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 podman[41406]: 2024-02-05 07:29:59.739833759 +0000 UTC m=+0.132249164 container create 223c6dab2a8206070db1eabae34b926e5691c570d168e8eab4f35cbd17d9d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 podman[41406]: 2024-02-05 07:29:59.917342945 +0000 UTC m=+0.309758309 container init 223c6dab2a8206070db1eabae34b926e5691c570d168e8eab4f35cbd17d9d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 podman[41406]: 2024-02-05 07:29:59.923175021 +0000 UTC m=+0.315590385 container start 223c6dab2a8206070db1eabae34b926e5691c570d168e8eab4f35cbd17d9d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: set uid:gid to 167:167 (ceph:ceph) 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev), process ceph-mon, pid 7 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pidfile_write: ignore empty --pid-file 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: load: jerasure load: lrc 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: RocksDB version: 7.9.2 2024-02-05T07:30:00.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Git sha 0 2024-02-05T07:30:00.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Compile date 2024-02-04 23:11:06 2024-02-05T07:30:00.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: DB SUMMARY 2024-02-05T07:30:00.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: DB Session ID: M2IIU4BM7829ERYUEPVS 2024-02-05T07:30:00.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: CURRENT file: CURRENT 2024-02-05T07:30:00.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: IDENTITY file: IDENTITY 2024-02-05T07:30:00.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-05T07:30:00.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.error_if_exists: 0 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.create_if_missing: 0 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.paranoid_checks: 1 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.env: 0x564c723deae0 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.fs: PosixFileSystem 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.info_log: 0x564c72ca18c0 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_file_opening_threads: 16 2024-02-05T07:30:00.013 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.statistics: (nil) 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.use_fsync: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_log_file_size: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.keep_log_file_num: 1000 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.recycle_log_file_num: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.allow_fallocate: 1 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.allow_mmap_reads: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.allow_mmap_writes: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.use_direct_reads: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.create_missing_column_families: 0 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.db_log_dir: 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.wal_dir: 2024-02-05T07:30:00.014 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.advise_random_on_open: 1 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.db_write_buffer_size: 0 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.write_buffer_manager: 0x564c72c75400 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.rate_limiter: (nil) 2024-02-05T07:30:00.015 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.wal_recovery_mode: 2 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.enable_thread_tracking: 0 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.enable_pipelined_write: 0 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.unordered_write: 0 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.row_cache: None 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.wal_filter: None 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.allow_ingest_behind: 0 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.two_write_queues: 0 2024-02-05T07:30:00.016 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.manual_wal_flush: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.wal_compression: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.atomic_flush: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.log_readahead_size: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.best_efforts_recovery: 0 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-05T07:30:00.017 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.allow_data_in_errors: 0 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.db_host_id: __hostname__ 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_background_jobs: 2 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_background_compactions: -1 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_subcompactions: 1 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_total_wal_size: 0 2024-02-05T07:30:00.018 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_open_files: -1 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bytes_per_sync: 0 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_readahead_size: 0 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_background_flushes: -1 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Compression algorithms supported: 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kZSTD supported: 0 2024-02-05T07:30:00.019 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kXpressCompression supported: 0 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kLZ4HCCompression supported: 1 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kZlibCompression supported: 1 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kSnappyCompression supported: 1 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kLZ4Compression supported: 1 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: kBZip2Compression supported: 0 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.merge_operator: 2024-02-05T07:30:00.020 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_filter: None 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_filter_factory: None 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.sst_partitioner_factory: None 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564c72ca19c0) 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_type: 0 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_index_type: 0 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_shortening: 1 2024-02-05T07:30:00.021 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: checksum: 4 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: no_block_cache: 0 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache: 0x564c72ca5090 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_options: 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: capacity : 536870912 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_shard_bits : 4 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: strict_capacity_limit : 0 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_compressed: (nil) 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: persistent_cache: (nil) 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size: 4096 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size_deviation: 10 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_restart_interval: 16 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_block_restart_interval: 1 2024-02-05T07:30:00.022 INFO:journalctl@ceph.mon.c.smithi043.stdout: metadata_block_size: 4096 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: partition_filters: 0 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: use_delta_encoding: 1 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: filter_policy: bloomfilter 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: whole_key_filtering: 1 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: verify_compression: 0 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: format_version: 5 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_index_compression: 1 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_align: 0 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: max_auto_readahead_size: 262144 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: prepopulate_block_cache: 0 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: initial_auto_readahead_size: 8192 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.write_buffer_size: 33554432 2024-02-05T07:30:00.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_write_buffer_number: 2 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression: NoCompression 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression: Disabled 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.prefix_extractor: nullptr 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.num_levels: 7 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-05T07:30:00.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.level: 32767 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.strategy: 0 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.enabled: false 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-05T07:30:00.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.target_file_size_base: 67108864 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-05T07:30:00.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.arena_block_size: 1048576 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.disable_auto_compactions: 0 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-05T07:30:00.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.table_properties_collectors: 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.inplace_update_support: 0 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-05T07:30:00.028 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.bloom_locality: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.max_successive_merges: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.paranoid_file_checks: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.force_consistency_checks: 1 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.report_bg_io_stats: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.ttl: 2592000 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.enable_blob_files: false 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.min_blob_size: 0 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.blob_file_size: 268435456 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-05T07:30:00.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.blob_file_starting_level: 0 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 085018dd-6b81-40ee-bfc7-691f00edbdef 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118199953075, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118199953466, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707118199, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "085018dd-6b81-40ee-bfc7-691f00edbdef", "db_session_id": "M2IIU4BM7829ERYUEPVS", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707118199953551, "job": 1, "event": "recovery_finished"} 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-05T07:30:00.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564c72d96000 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: DB pointer 0x564c72d80000 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** DB Stats ** 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-05T07:30:00.031 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-02-05T07:30:00.032 INFO:journalctl@ceph.mon.c.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache BinnedLRUCache@0x564c72ca5090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-05T07:30:00.033 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: using public_addrv [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: starting mon.c rank -1 at public addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] at bind addrs [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(???) e0 preinit fsid f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).mds e1 new map 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout: e1 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout: legacy client fscid: -1 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout: No filesystems configured 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-05T07:30:00.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/927673057' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/927673057' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:00.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:00.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:00.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:00.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:00.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.037 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: Saving service mon spec with placement smithi043:172.21.15.43=a;smithi043:[v2:172.21.15.43:3301,v1:172.21.15.43:6790]=c;smithi050:172.21.15.50=b;count:3 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: Deploying daemon mon.b on smithi050 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/2759718565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.a calling monitor election 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.b calling monitor election 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:00.038 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: monmap e2: 2 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: fsmap 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mgrmap e13: y(active, since 41s) 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: overall HEALTH_OK 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:00.039 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: Deploying daemon mon.c on smithi043 2024-02-05T07:30:00.040 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/3127904970' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:30:00.040 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:00.040 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:00.040 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:29:59 smithi043 ceph-mon[41444]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-05T07:30:00.040 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:00 smithi043 bash[41406]: 223c6dab2a8206070db1eabae34b926e5691c570d168e8eab4f35cbd17d9d2c1 2024-02-05T07:30:00.040 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:00 smithi043 systemd[1]: Started Ceph mon.c for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: mon.a calling monitor election 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: mon.b calling monitor election 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: mon.c calling monitor election 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: fsmap 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: mgrmap e13: y(active, since 49s) 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: overall HEALTH_OK 2024-02-05T07:30:05.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.326 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.326 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.326 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: mon.a calling monitor election 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: mon.b calling monitor election 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: mon.c calling monitor election 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: fsmap 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: mgrmap e13: y(active, since 49s) 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: overall HEALTH_OK 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:30:05.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: mon.a calling monitor election 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: mon.b calling monitor election 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: mon.c calling monitor election 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: monmap e3: 3 mons at {a=[v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0],b=[v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0],c=[v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: fsmap 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: osdmap e4: 0 total, 0 up, 0 in 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: mgrmap e13: y(active, since 49s) 2024-02-05T07:30:05.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: overall HEALTH_OK 2024-02-05T07:30:05.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:05.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:06 smithi050 ceph-mon[33492]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:06.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.534+0000 7f39cd398700 1 -- 172.21.15.50:0/1729070076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b4006240 msgr2=0x7f39b40086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:06.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.534+0000 7f39cd398700 1 --2- 172.21.15.50:0/1729070076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b4006240 0x7f39b40086b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f39c806a0c0 tx=0x7f39c00095d0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.534+0000 7f39cd398700 1 -- 172.21.15.50:0/1729070076 shutdown_connections 2024-02-05T07:30:06.537 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.534+0000 7f39cd398700 1 --2- 172.21.15.50:0/1729070076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39b4006240 0x7f39b40086b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.534+0000 7f39cd398700 1 -- 172.21.15.50:0/1729070076 >> 172.21.15.50:0/1729070076 conn(0x7f39c80fc1f0 msgr2=0x7f39c80fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.535+0000 7f39cd398700 1 -- 172.21.15.50:0/1729070076 shutdown_connections 2024-02-05T07:30:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.535+0000 7f39cd398700 1 -- 172.21.15.50:0/1729070076 wait complete. 2024-02-05T07:30:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.535+0000 7f39cd398700 1 Processor -- start 2024-02-05T07:30:06.538 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.535+0000 7f39cd398700 1 -- start start 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39cd398700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b4006240 0x7f39c819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39cd398700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39c819cb10 0x7f39c81966a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39cd398700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39c8196be0 0x7f39c8197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39cd398700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c819d0f0 con 0x7f39c819cb10 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39cd398700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c81a3380 con 0x7f39b4006240 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39cd398700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c81a3510 con 0x7f39c8196be0 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b4006240 0x7f39c819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b4006240 0x7f39c819c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:38870/0 (socket says 172.21.15.50:38870) 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 -- 172.21.15.50:0/4053728311 learned_addr learned my addr 172.21.15.50:0/4053728311 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c67fc700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39c819cb10 0x7f39c81966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c77fe700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39c8196be0 0x7f39c8197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 -- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39c8196be0 msgr2=0x7f39c8197090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:06.540 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39c8196be0 0x7f39c8197090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 -- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39c819cb10 msgr2=0x7f39c81966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39c819cb10 0x7f39c81966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 -- 172.21.15.50:0/4053728311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c00098d0 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39c6ffd700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b4006240 0x7f39c819c5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009280 tx=0x7f39c0004250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.536+0000 7f39bffff700 1 -- 172.21.15.50:0/4053728311 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0003e10 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.537+0000 7f39bffff700 1 -- 172.21.15.50:0/4053728311 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f39c001a8c0 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.537+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c81a3790 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.537+0000 7f39bffff700 1 -- 172.21.15.50:0/4053728311 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c0019830 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.537+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c81a3c80 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.537+0000 7f39bffff700 1 -- 172.21.15.50:0/4053728311 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f39c0019990 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.538+0000 7f39bffff700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f39a8039410 0x7f39a803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.538+0000 7f39bffff700 1 -- 172.21.15.50:0/4053728311 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f39c004e2c0 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.538+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c806a740 con 0x7f39b4006240 2024-02-05T07:30:06.541 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.538+0000 7f39c67fc700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f39a8039410 0x7f39a803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:06.542 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.538+0000 7f39c67fc700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f39a8039410 0x7f39a803b8d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009ad0 tx=0x7f39b0009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:06.545 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.543+0000 7f39bffff700 1 -- 172.21.15.50:0/4053728311 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39c0034350 con 0x7f39b4006240 2024-02-05T07:30:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[41444]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[29879]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:30:06.835 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.833+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f39c8197bd0 con 0x7f39b4006240 2024-02-05T07:30:06.836 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.833+0000 7f39bffff700 1 -- 172.21.15.50:0/4053728311 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1277 (secure 0 0 0) 0x7f39c0017c40 con 0x7f39b4006240 2024-02-05T07:30:06.836 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-02-05T07:30:06.836 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"fsid":"f211dd04-c3f7-11ee-95b6-87774f69a715","modified":"2024-02-05T07:29:59.973901Z","created":"2024-02-05T07:28:11.879318Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3300","nonce":0},{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:3301","nonce":0},{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-05T07:30:06.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f39a8039410 msgr2=0x7f39a803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:06.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f39a8039410 0x7f39a803b8d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009ad0 tx=0x7f39b0009210 comp rx=0 tx=0).stop 2024-02-05T07:30:06.839 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b4006240 msgr2=0x7f39c819c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b4006240 0x7f39c819c5d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009280 tx=0x7f39c0004250 comp rx=0 tx=0).stop 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 shutdown_connections 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f39a8039410 0x7f39a803b8d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f39c8196be0 0x7f39c8197090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f39b4006240 0x7f39c819c5d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 --2- 172.21.15.50:0/4053728311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f39c819cb10 0x7f39c81966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.835+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 >> 172.21.15.50:0/4053728311 conn(0x7f39c80fc1f0 msgr2=0x7f39c81062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.836+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 shutdown_connections 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:06.836+0000 7f39cd398700 1 -- 172.21.15.50:0/4053728311 wait complete. 2024-02-05T07:30:06.840 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 3 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: Updating smithi043:/etc/ceph/ceph.conf 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:30:07.487 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/4053728311' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:07.488 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:07.514 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-05T07:30:07.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph config generate-minimal-conf 2024-02-05T07:30:07.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: Updating smithi043:/etc/ceph/ceph.conf 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/4053728311' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: Updating smithi043:/etc/ceph/ceph.conf 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/4053728311' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.709 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:07.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:07.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:07.710 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:30:08.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[29879]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:08.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[29879]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T07:30:08.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[29879]: Reconfiguring daemon mon.a on smithi043 2024-02-05T07:30:08.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:08.723 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:08.723 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[41444]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:08.723 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[41444]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T07:30:08.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[41444]: Reconfiguring daemon mon.a on smithi043 2024-02-05T07:30:08.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:08.724 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:08 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:08 smithi050 ceph-mon[33492]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:08 smithi050 ceph-mon[33492]: Reconfiguring mon.a (unknown last config time)... 2024-02-05T07:30:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:08 smithi050 ceph-mon[33492]: Reconfiguring daemon mon.a on smithi043 2024-02-05T07:30:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:08 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:08 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[29879]: Reconfiguring mon.c (monmap changed)... 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[29879]: Reconfiguring daemon mon.c on smithi043 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[41444]: Reconfiguring mon.c (monmap changed)... 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:09.780 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:09.781 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:09 smithi043 ceph-mon[41444]: Reconfiguring daemon mon.c on smithi043 2024-02-05T07:30:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:09 smithi050 ceph-mon[33492]: Reconfiguring mon.c (monmap changed)... 2024-02-05T07:30:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:09 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:09 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:09 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:09 smithi050 ceph-mon[33492]: Reconfiguring daemon mon.c on smithi043 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 -- 172.21.15.43:0/772086217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 msgr2=0x7fdf68107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/772086217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 0x7fdf68107040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c009990 tx=0x7fdf5c009ca0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 -- 172.21.15.43:0/772086217 shutdown_connections 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/772086217 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf681015c0 0x7fdf6810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/772086217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 0x7fdf68107040 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/772086217 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf68100c20 0x7fdf68101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 -- 172.21.15.43:0/772086217 >> 172.21.15.43:0/772086217 conn(0x7fdf68077b50 msgr2=0x7fdf68077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:11.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 -- 172.21.15.43:0/772086217 shutdown_connections 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.202+0000 7fdf6ed88700 1 -- 172.21.15.43:0/772086217 wait complete. 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 Processor -- start 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 -- start start 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf68100c20 0x7fdf6819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf681015c0 0x7fdf6819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 0x7fdf681975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68197c00 con 0x7fdf68106c60 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68197d40 con 0x7fdf68100c20 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6ed88700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf68197f00 con 0x7fdf681015c0 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf681015c0 0x7fdf6819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:11.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6d325700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 0x7fdf681975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf681015c0 0x7fdf6819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:57464/0 (socket says 172.21.15.43:57464) 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 -- 172.21.15.43:0/2882643251 learned_addr learned my addr 172.21.15.43:0/2882643251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6cb24700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf68100c20 0x7fdf6819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 -- 172.21.15.43:0/2882643251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf68100c20 msgr2=0x7fdf6819d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf68100c20 0x7fdf6819d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 -- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 msgr2=0x7fdf681975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 0x7fdf681975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 -- 172.21.15.43:0/2882643251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf54009520 con 0x7fdf681015c0 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf6d325700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 0x7fdf681975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.203+0000 7fdf67fff700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf681015c0 0x7fdf6819d920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400eb30 tx=0x7fdf5400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:11.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.204+0000 7fdf65ffb700 1 -- 172.21.15.43:0/2882643251 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54009880 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.204+0000 7fdf65ffb700 1 -- 172.21.15.43:0/2882643251 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdf540044a0 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.204+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c0095f0 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.204+0000 7fdf65ffb700 1 -- 172.21.15.43:0/2882643251 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf54003ca0 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.204+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf681a42b0 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.205+0000 7fdf65ffb700 1 -- 172.21.15.43:0/2882643251 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7fdf54018430 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.205+0000 7fdf65ffb700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fdf580391c0 0x7fdf5803b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.205+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf68109160 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.205+0000 7fdf65ffb700 1 -- 172.21.15.43:0/2882643251 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fdf54014070 con 0x7fdf681015c0 2024-02-05T07:30:11.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.205+0000 7fdf6cb24700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fdf580391c0 0x7fdf5803b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:11.041 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:30:11.041 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-02-05T07:30:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:30:11.041 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] 2024-02-05T07:30:11.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.205+0000 7fdf6cb24700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fdf580391c0 0x7fdf5803b680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00b2e0 tx=0x7fdf5c005040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:11.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.210+0000 7fdf65ffb700 1 -- 172.21.15.43:0/2882643251 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf54020940 con 0x7fdf681015c0 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.421+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fdf68198ad0 con 0x7fdf681015c0 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.422+0000 7fdf65ffb700 1 -- 172.21.15.43:0/2882643251 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7fdf54020710 con 0x7fdf681015c0 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fdf580391c0 msgr2=0x7fdf5803b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fdf580391c0 0x7fdf5803b680 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00b2e0 tx=0x7fdf5c005040 comp rx=0 tx=0).stop 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf681015c0 msgr2=0x7fdf6819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf681015c0 0x7fdf6819d920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400eb30 tx=0x7fdf5400ee40 comp rx=0 tx=0).stop 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 shutdown_connections 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fdf580391c0 0x7fdf5803b680 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf68106c60 0x7fdf681975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdf68100c20 0x7fdf6819d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 --2- 172.21.15.43:0/2882643251 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdf681015c0 0x7fdf6819d920 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:11.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 >> 172.21.15.43:0/2882643251 conn(0x7fdf68077b50 msgr2=0x7fdf680ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:11.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 shutdown_connections 2024-02-05T07:30:11.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:10.424+0000 7fdf6ed88700 1 -- 172.21.15.43:0/2882643251 wait complete. 2024-02-05T07:30:11.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:10 smithi043 ceph-mon[29879]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:11.043 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2882643251' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:11.043 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:10 smithi043 ceph-mon[41444]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:11.044 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2882643251' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:10 smithi050 ceph-mon[33492]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2882643251' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:11.551 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-05T07:30:11.551 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:30:11.552 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-02-05T07:30:11.582 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:30:11.582 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:30:11.649 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:30:11.649 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-02-05T07:30:11.679 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:30:11.680 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:30:11.746 INFO:tasks.cephadm:Adding mgr.y on smithi043 2024-02-05T07:30:11.746 INFO:tasks.cephadm:Adding mgr.x on smithi050 2024-02-05T07:30:11.746 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply mgr '2;smithi043=y;smithi050=x' 2024-02-05T07:30:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:11 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:11 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:11 smithi050 ceph-mon[33492]: Reconfiguring mon.b (monmap changed)... 2024-02-05T07:30:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:11 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:11 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:11 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:11 smithi050 ceph-mon[33492]: Reconfiguring daemon mon.b on smithi050 2024-02-05T07:30:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[29879]: Reconfiguring mon.b (monmap changed)... 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[29879]: Reconfiguring daemon mon.b on smithi050 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[41444]: Reconfiguring mon.b (monmap changed)... 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:11 smithi043 ceph-mon[41444]: Reconfiguring daemon mon.b on smithi050 2024-02-05T07:30:12.590 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:30:13.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:12 smithi050 ceph-mon[33492]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:13.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:13.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:13.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:13.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[29879]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[41444]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:14.121 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.118+0000 7ff7598b2700 1 -- 172.21.15.50:0/552208296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541015f0 msgr2=0x7ff754101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.118+0000 7ff7598b2700 1 --2- 172.21.15.50:0/552208296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541015f0 0x7ff754101a50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff7480099c0 tx=0x7ff748009cd0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.118+0000 7ff7598b2700 1 -- 172.21.15.50:0/552208296 shutdown_connections 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.118+0000 7ff7598b2700 1 --2- 172.21.15.50:0/552208296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff754101f90 0x7ff754108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.118+0000 7ff7598b2700 1 --2- 172.21.15.50:0/552208296 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff7541003f0 0x7ff754100810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.118+0000 7ff7598b2700 1 --2- 172.21.15.50:0/552208296 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541015f0 0x7ff754101a50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.118+0000 7ff7598b2700 1 -- 172.21.15.50:0/552208296 >> 172.21.15.50:0/552208296 conn(0x7ff7540fb9b0 msgr2=0x7ff7540fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 -- 172.21.15.50:0/552208296 shutdown_connections 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 -- 172.21.15.50:0/552208296 wait complete. 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 Processor -- start 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 -- start start 2024-02-05T07:30:14.122 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541003f0 0x7ff754197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7541015f0 0x7ff754197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff754101f90 0x7ff75419e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75419eee0 con 0x7ff7541015f0 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75419f020 con 0x7ff7541003f0 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.119+0000 7ff7598b2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75419f1b0 con 0x7ff754101f90 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541003f0 0x7ff754197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541003f0 0x7ff754197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:43926/0 (socket says 172.21.15.50:43926) 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 -- 172.21.15.50:0/3569643874 learned_addr learned my addr 172.21.15.50:0/3569643874 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff7537fe700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff754101f90 0x7ff75419e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff7527fc700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7541015f0 0x7ff754197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 -- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff754101f90 msgr2=0x7ff75419e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff754101f90 0x7ff75419e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 -- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7541015f0 msgr2=0x7ff754197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7541015f0 0x7ff754197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.123 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 -- 172.21.15.50:0/3569643874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7480095f0 con 0x7ff7541003f0 2024-02-05T07:30:14.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff752ffd700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541003f0 0x7ff754197350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b500 tx=0x7ff74400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:14.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff7588b0700 1 -- 172.21.15.50:0/3569643874 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744011830 con 0x7ff7541003f0 2024-02-05T07:30:14.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff7588b0700 1 -- 172.21.15.50:0/3569643874 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff744011e10 con 0x7ff7541003f0 2024-02-05T07:30:14.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75419f490 con 0x7ff7541003f0 2024-02-05T07:30:14.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff7588b0700 1 -- 172.21.15.50:0/3569643874 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744010c60 con 0x7ff7541003f0 2024-02-05T07:30:14.124 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.120+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75419fa60 con 0x7ff7541003f0 2024-02-05T07:30:14.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.121+0000 7ff7588b0700 1 -- 172.21.15.50:0/3569643874 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7ff74401a400 con 0x7ff7541003f0 2024-02-05T07:30:14.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.121+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7540689e0 con 0x7ff7541003f0 2024-02-05T07:30:14.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.121+0000 7ff7588b0700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff73c0391c0 0x7ff73c03b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:14.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.121+0000 7ff7588b0700 1 -- 172.21.15.50:0/3569643874 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff74404b620 con 0x7ff7541003f0 2024-02-05T07:30:14.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.122+0000 7ff7527fc700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff73c0391c0 0x7ff73c03b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:14.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.122+0000 7ff7527fc700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff73c0391c0 0x7ff73c03b680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff748009990 tx=0x7ff74800b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:14.127 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.126+0000 7ff7588b0700 1 -- 172.21.15.50:0/3569643874 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff744010e00 con 0x7ff7541003f0 2024-02-05T07:30:14.339 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.338+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi050=x", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7540fdbc0 con 0x7ff73c0391c0 2024-02-05T07:30:14.345 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.344+0000 7ff7588b0700 1 -- 172.21.15.50:0/3569643874 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7ff7540fdbc0 con 0x7ff73c0391c0 2024-02-05T07:30:14.345 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled mgr update... 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff73c0391c0 msgr2=0x7ff73c03b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff73c0391c0 0x7ff73c03b680 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff748009990 tx=0x7ff74800b790 comp rx=0 tx=0).stop 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541003f0 msgr2=0x7ff754197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541003f0 0x7ff754197350 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ff74400b500 tx=0x7ff74400b810 comp rx=0 tx=0).stop 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 shutdown_connections 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff73c0391c0 0x7ff73c03b680 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff754101f90 0x7ff75419e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.348 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff7541003f0 0x7ff754197350 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.346+0000 7ff7598b2700 1 --2- 172.21.15.50:0/3569643874 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7541015f0 0x7ff754197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:14.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.347+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 >> 172.21.15.50:0/3569643874 conn(0x7ff7540fb9b0 msgr2=0x7ff7540fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:14.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.347+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 shutdown_connections 2024-02-05T07:30:14.349 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:30:14.347+0000 7ff7598b2700 1 -- 172.21.15.50:0/3569643874 wait complete. 2024-02-05T07:30:14.907 DEBUG:teuthology.orchestra.run.smithi050:mgr.x> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.x.service 2024-02-05T07:30:14.909 INFO:tasks.cephadm:Deploying OSDs... 2024-02-05T07:30:14.910 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:30:14.910 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T07:30:14.929 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-05T07:30:14.929 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-02-05 07:29:59.451237345 +0000 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-02-05 07:24:04.846106596 +0000 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-02-05 07:24:04.846106596 +0000 2024-02-05T07:30:14.987 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-02-05 07:24:04.846106596 +0000 2024-02-05T07:30:14.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-05T07:30:15.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-02-05T07:30:15.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-02-05T07:30:15.057 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000137579 s, 3.7 MB/s 2024-02-05T07:30:15.059 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-05T07:30:15.117 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-02-05T07:30:15.173 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-02-05 07:29:59.452237332 +0000 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-02-05 07:24:05.174102469 +0000 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-02-05 07:24:05.174102469 +0000 2024-02-05T07:30:15.174 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-02-05 07:24:05.174102469 +0000 2024-02-05T07:30:15.174 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T07:30:15.240 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:30:15.241 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:15.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-02-05T07:30:15.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-02-05T07:30:15.242 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.00011741 s, 4.4 MB/s 2024-02-05T07:30:15.243 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-05T07:30:15.291 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:30:15.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:15.300 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-02-05 07:29:59.452237332 +0000 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-02-05 07:24:05.510098240 +0000 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-02-05 07:24:05.510098240 +0000 2024-02-05T07:30:15.356 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-02-05 07:24:05.510098240 +0000 2024-02-05T07:30:15.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-05T07:30:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-02-05T07:30:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-02-05T07:30:15.423 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000109604 s, 4.7 MB/s 2024-02-05T07:30:15.424 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-05T07:30:15.481 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-02-05T07:30:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-05T07:30:15.537 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:15.537 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-05T07:30:15.538 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:15.538 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:15.538 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-02-05 07:29:59.452237332 +0000 2024-02-05T07:30:15.538 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-02-05 07:24:05.842094062 +0000 2024-02-05T07:30:15.538 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-02-05 07:24:05.842094062 +0000 2024-02-05T07:30:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-02-05 07:24:05.842094062 +0000 2024-02-05T07:30:15.538 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-05T07:30:15.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-02-05T07:30:15.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-02-05T07:30:15.603 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 8.5857e-05 s, 6.0 MB/s 2024-02-05T07:30:15.604 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-05T07:30:15.660 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:30:15.660 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-02-05T07:30:15.675 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-05T07:30:15.675 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-02-05T07:30:15.730 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-05 07:29:50.352906898 +0000 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-05 07:24:04.854070670 +0000 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-05 07:24:04.854070670 +0000 2024-02-05T07:30:15.731 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-05 07:24:04.854070670 +0000 2024-02-05T07:30:15.731 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-05T07:30:15.797 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-05T07:30:15.797 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-05T07:30:15.797 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000101772 s, 5.0 MB/s 2024-02-05T07:30:15.798 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-05T07:30:15.854 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-05 07:29:50.352906898 +0000 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-05 07:24:05.196062149 +0000 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-05 07:24:05.196062149 +0000 2024-02-05T07:30:15.909 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-05 07:24:05.196062149 +0000 2024-02-05T07:30:15.910 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-05T07:30:15.974 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-05T07:30:15.974 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-05T07:30:15.974 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.8752e-05 s, 5.2 MB/s 2024-02-05T07:30:15.975 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-05T07:30:16.031 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-02-05T07:30:16.065 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:15 smithi050 ceph-mon[33492]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi050=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:16.065 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:15 smithi050 ceph-mon[33492]: Saving service mgr spec with placement smithi043=y;smithi050=x;count:2 2024-02-05T07:30:16.065 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:15 smithi050 ceph-mon[33492]: Deploying daemon mgr.x on smithi050 2024-02-05T07:30:16.079 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-05T07:30:16.079 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:16.080 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-05T07:30:16.080 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:16.080 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:16.080 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-05 07:29:50.353906873 +0000 2024-02-05T07:30:16.080 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-05 07:24:05.532053778 +0000 2024-02-05T07:30:16.080 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-05 07:24:05.532053778 +0000 2024-02-05T07:30:16.080 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-05 07:24:05.532053778 +0000 2024-02-05T07:30:16.080 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-05T07:30:16.145 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-05T07:30:16.145 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-05T07:30:16.145 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000101534 s, 5.0 MB/s 2024-02-05T07:30:16.146 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-05T07:30:16.206 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-02-05T07:30:16.261 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-05T07:30:16.261 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-05T07:30:16.261 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-05T07:30:16.261 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-05T07:30:16.261 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-02-05T07:30:16.261 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-02-05 07:29:50.353906873 +0000 2024-02-05T07:30:16.261 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-02-05 07:24:05.868045406 +0000 2024-02-05T07:30:16.262 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-02-05 07:24:05.868045406 +0000 2024-02-05T07:30:16.262 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-02-05 07:24:05.868045406 +0000 2024-02-05T07:30:16.262 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-05T07:30:16.300 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:15 smithi043 ceph-mon[29879]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi050=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:16.300 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:15 smithi043 ceph-mon[29879]: Saving service mgr spec with placement smithi043=y;smithi050=x;count:2 2024-02-05T07:30:16.301 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:15 smithi043 ceph-mon[29879]: Deploying daemon mgr.x on smithi050 2024-02-05T07:30:16.301 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:15 smithi043 ceph-mon[41444]: from='client.24112 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi050=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:16.301 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:15 smithi043 ceph-mon[41444]: Saving service mgr spec with placement smithi043=y;smithi050=x;count:2 2024-02-05T07:30:16.301 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:15 smithi043 ceph-mon[41444]: Deploying daemon mgr.x on smithi050 2024-02-05T07:30:16.329 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-02-05T07:30:16.329 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-02-05T07:30:16.329 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 9.7176e-05 s, 5.3 MB/s 2024-02-05T07:30:16.330 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-05T07:30:16.387 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-02-05T07:30:16.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-05T07:30:16.632 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:30:16.707 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 systemd[1]: Starting Ceph mgr.x for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:30:16.962 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 podman[35585]: 2024-02-05 07:30:16.705690335 +0000 UTC m=+0.103931265 container create 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T07:30:16.962 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 podman[35585]: 2024-02-05 07:30:16.619621303 +0000 UTC m=+0.017862196 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T07:30:16.962 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 podman[35585]: 2024-02-05 07:30:16.908373974 +0000 UTC m=+0.306614859 container init 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2) 2024-02-05T07:30:16.962 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 podman[35585]: 2024-02-05 07:30:16.913852508 +0000 UTC m=+0.312093395 container start 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2) 2024-02-05T07:30:16.962 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 ceph-mgr[35623]: -- 172.21.15.50:0/3149375043 <== mon.1 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a4a37b1520 con 0x55a4a3786c00 2024-02-05T07:30:17.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:16 smithi043 ceph-mon[29879]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:17.010 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:16 smithi043 ceph-mon[41444]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:16 smithi050 ceph-mon[33492]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:17.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 bash[35585]: 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 2024-02-05T07:30:17.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:16 smithi050 systemd[1]: Started Ceph mgr.x for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:30:17.823 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:17.487+0000 7f23294d7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:30:17.823 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:17.584+0000 7f23294d7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:30:18.278 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-mon[33492]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:18.278 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.278 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.278 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.278 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.278 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[41444]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:18.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[29879]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:18.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:18.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:18.573 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:18.276+0000 7f23294d7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:30:18.573 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:18.376+0000 7f23294d7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:30:18.573 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:18.521+0000 7f23294d7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:30:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-05T07:30:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-05T07:30:18.616 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-02-05T07:30:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-02-05T07:30:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00859952 s, 1.2 GB/s 2024-02-05T07:30:18.616 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 2024-02-05T07:30:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-02-05T07:30:18.971 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:18.619+0000 7f23294d7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:30:18.971 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:18.703+0000 7f23294d7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:30:18.971 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:18.970+0000 7f23294d7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:30:19.221 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-02-05T07:30:19.266 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:19.082+0000 7f23294d7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:30:19.266 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:19.169+0000 7f23294d7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:30:19.267 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:19.265+0000 7f23294d7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:30:19.505 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.505 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.505 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:19.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:19.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T07:30:19.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:30:19.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:30:19.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:19.507 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[41444]: Reconfiguring daemon mgr.y on smithi043 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:19.508 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:19 smithi043 ceph-mon[29879]: Reconfiguring daemon mgr.y on smithi043 2024-02-05T07:30:19.573 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:19.443+0000 7f23294d7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:30:19.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:19.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:19.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:19.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: Reconfiguring mgr.y (unknown last config time)... 2024-02-05T07:30:19.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-05T07:30:19.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-05T07:30:19.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:19.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:19 smithi050 ceph-mon[33492]: Reconfiguring daemon mgr.y on smithi043 2024-02-05T07:30:19.950 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:30:20.823 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:20.445+0000 7f23294d7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:30:20.823 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:20.579+0000 7f23294d7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:30:21.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[41444]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:21.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:21.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[29879]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:21.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-mon[33492]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:20.918+0000 7f23294d7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:21.009+0000 7f23294d7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:30:21.210 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:21.208+0000 7f23294d7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:30:21.573 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:21.396+0000 7f23294d7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:30:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 -- 172.21.15.43:0/784466470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04101e50 msgr2=0x7f1a041022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/784466470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04101e50 0x7f1a041022b0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f19f00098f0 tx=0x7f19f0009c00 comp rx=0 tx=0).stop 2024-02-05T07:30:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 -- 172.21.15.43:0/784466470 shutdown_connections 2024-02-05T07:30:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/784466470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a041027f0 0x7f1a041090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/784466470 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a04100c50 0x7f1a04101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:21.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/784466470 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04101e50 0x7f1a041022b0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 -- 172.21.15.43:0/784466470 >> 172.21.15.43:0/784466470 conn(0x7f1a040fc1f0 msgr2=0x7f1a040fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 -- 172.21.15.43:0/784466470 shutdown_connections 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.877+0000 7f1a0acbe700 1 -- 172.21.15.43:0/784466470 wait complete. 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 Processor -- start 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 -- start start 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04100c50 0x7f1a04104be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a04101e50 0x7f1a04103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a041027f0 0x7f1a04103770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04103ef0 con 0x7f1a04101e50 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a04104060 con 0x7f1a041027f0 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0acbe700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a041a1350 con 0x7f1a04100c50 2024-02-05T07:30:21.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a03fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a04101e50 0x7f1a04103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a03fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a04101e50 0x7f1a04103230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52350/0 (socket says 172.21.15.43:52350) 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a08a5a700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04100c50 0x7f1a04104be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a03fff700 1 -- 172.21.15.43:0/605415237 learned_addr learned my addr 172.21.15.43:0/605415237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0925b700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a041027f0 0x7f1a04103770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a08a5a700 1 -- 172.21.15.43:0/605415237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a041027f0 msgr2=0x7f1a04103770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a08a5a700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a041027f0 0x7f1a04103770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a08a5a700 1 -- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a04101e50 msgr2=0x7f1a04103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a08a5a700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a04101e50 0x7f1a04103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a08a5a700 1 -- 172.21.15.43:0/605415237 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f80095f0 con 0x7f1a04100c50 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a03fff700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a04101e50 0x7f1a04103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.878+0000 7f1a0925b700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a041027f0 0x7f1a04103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:30:21.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.879+0000 7f1a08a5a700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04100c50 0x7f1a04104be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005fd0 tx=0x7f19f800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.879+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f800cbd0 con 0x7f1a04100c50 2024-02-05T07:30:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.879+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc009c40 con 0x7f1a04100c50 2024-02-05T07:30:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.879+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f19f800cd30 con 0x7f1a04100c50 2024-02-05T07:30:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.879+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f8018680 con 0x7f1a04100c50 2024-02-05T07:30:21.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.879+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f0009520 con 0x7f1a04100c50 2024-02-05T07:30:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.880+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 13) v1 ==== 45988+0+0 (secure 0 0 0) 0x7f19f80187e0 con 0x7f1a04100c50 2024-02-05T07:30:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.880+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a04108a30 con 0x7f1a04100c50 2024-02-05T07:30:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.880+0000 7f1a01ffb700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f19f40391c0 0x7f19f403b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.880+0000 7f1a03fff700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f19f40391c0 0x7f19f403b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.881+0000 7f1a03fff700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f19f40391c0 0x7f19f403b680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009fd0 tx=0x7f19f0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:21.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.881+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f19f8014070 con 0x7f1a04100c50 2024-02-05T07:30:21.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:21.886+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19f8021910 con 0x7f1a04100c50 2024-02-05T07:30:21.948 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:21.684+0000 7f23294d7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:30:21.948 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:21.947+0000 7f23294d7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:30:22.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:22.102+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a04062160 con 0x7f19f40391c0 2024-02-05T07:30:22.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:22.188+0000 7f23294d7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:30:22.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[29879]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:22.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:22.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:22.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:22.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:30:22.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[41444]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:30:22.982 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:23.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:23.009+0000 7f23294d7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:30:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-mon[33492]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:30:23.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:30:23.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:23.887 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:23.591+0000 7f23294d7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:30:23.887 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:23.691+0000 7f23294d7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:30:24.030 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:23 smithi043 ceph-mon[29879]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:24.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:23 smithi043 ceph-mon[41444]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:23 smithi050 ceph-mon[33492]: from='client.24122 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:24.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:30:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:30:23.911+0000 7f23294d7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:30:24.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:24.892+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f19f80213f0 con 0x7f1a04100c50 2024-02-05T07:30:25.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[29879]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:25.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[29879]: Standby manager daemon x started 2024-02-05T07:30:25.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:30:25.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:30:25.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:30:25.260 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:30:25.261 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[41444]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:25.261 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[41444]: Standby manager daemon x started 2024-02-05T07:30:25.261 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:30:25.261 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:30:25.261 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:30:25.261 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:24 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:30:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:24 smithi050 ceph-mon[33492]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:24 smithi050 ceph-mon[33492]: Standby manager daemon x started 2024-02-05T07:30:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:24 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:30:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:24 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:30:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:24 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:30:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:24 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/1074895268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:30:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: mgrmap e14: y(active, since 69s), standbys: x 2024-02-05T07:30:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1625666346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]': finished 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3080033943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: mgrmap e14: y(active, since 69s), standbys: x 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1625666346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]: dispatch 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]': finished 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T07:30:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:26.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3080033943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: mgrmap e14: y(active, since 69s), standbys: x 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1625666346' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]: dispatch 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]: dispatch 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0c0b3234-6471-45a1-ad20-a2e96fd69264"}]': finished 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: osdmap e5: 1 total, 0 up, 1 in 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3080033943' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:30:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:26 smithi043 ceph-mon[29879]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:26 smithi043 ceph-mon[41444]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:26 smithi050 ceph-mon[33492]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:29.175 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:28 smithi043 ceph-mon[29879]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:29.175 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:28 smithi043 ceph-mon[41444]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:29.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:28 smithi050 ceph-mon[33492]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:31.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:30 smithi043 ceph-mon[29879]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:31.023 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:30 smithi043 ceph-mon[41444]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:30 smithi050 ceph-mon[33492]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:32.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T07:30:32.184 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:31 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:32.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:31 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T07:30:32.185 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:31 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:31 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-05T07:30:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:31 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:33.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:32 smithi043 ceph-mon[41444]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:33.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:32 smithi043 ceph-mon[41444]: Deploying daemon osd.0 on smithi043 2024-02-05T07:30:33.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:32 smithi043 ceph-mon[29879]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:33.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:32 smithi043 ceph-mon[29879]: Deploying daemon osd.0 on smithi043 2024-02-05T07:30:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:32 smithi050 ceph-mon[33492]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:32 smithi050 ceph-mon[33492]: Deploying daemon osd.0 on smithi043 2024-02-05T07:30:35.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:34 smithi043 ceph-mon[29879]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:35.192 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:34 smithi043 ceph-mon[41444]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:34 smithi050 ceph-mon[33492]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:37.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[29879]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:37.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:37.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:37.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:37.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[41444]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:37.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:37.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:37.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:36 smithi050 ceph-mon[33492]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:39.100 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:38 smithi043 ceph-mon[29879]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:39.100 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:38 smithi043 ceph-mon[41444]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:38 smithi050 ceph-mon[33492]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:40.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.555+0000 7f1a01ffb700 1 -- 172.21.15.43:0/605415237 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1a04062160 con 0x7f19f40391c0 2024-02-05T07:30:40.558 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-02-05T07:30:40.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f19f40391c0 msgr2=0x7f19f403b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f19f40391c0 0x7f19f403b680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009fd0 tx=0x7f19f0019040 comp rx=0 tx=0).stop 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04100c50 msgr2=0x7f1a04104be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04100c50 0x7f1a04104be0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f19f8005fd0 tx=0x7f19f800ee70 comp rx=0 tx=0).stop 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 shutdown_connections 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f19f40391c0 0x7f19f403b680 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a041027f0 0x7f1a04103770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a04100c50 0x7f1a04104be0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 --2- 172.21.15.43:0/605415237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a04101e50 0x7f1a04103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 >> 172.21.15.43:0/605415237 conn(0x7f1a040fc1f0 msgr2=0x7f1a040ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 shutdown_connections 2024-02-05T07:30:40.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:40.558+0000 7f1a0acbe700 1 -- 172.21.15.43:0/605415237 wait complete. 2024-02-05T07:30:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:40.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:40.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:30:40.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:30:40.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:40.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:30:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-05T07:30:41.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:41.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:41.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:30:41.219 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.0.service 2024-02-05T07:30:41.221 INFO:tasks.cephadm:Deploying osd.1 on smithi043 with /dev/vg_nvme/lv_3... 2024-02-05T07:30:41.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-05T07:30:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[41444]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[41444]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[41444]: from='osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[41444]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[29879]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[29879]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T07:30:42.203 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:42.204 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[29879]: from='osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:30:42.204 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-mon[29879]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:30:42.204 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0[45879]: 2024-02-05T07:30:41.937+0000 7f82d5f62700 -1 osd.0 0 waiting for initial osdmap 2024-02-05T07:30:42.204 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:30:41 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0[45879]: 2024-02-05T07:30:41.939+0000 7f82d1d5c700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:30:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:41 smithi050 ceph-mon[33492]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-05T07:30:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:41 smithi050 ceph-mon[33492]: osdmap e6: 1 total, 0 up, 1 in 2024-02-05T07:30:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:41 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:41 smithi050 ceph-mon[33492]: from='osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:30:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:41 smithi050 ceph-mon[33492]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:30:43.299 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[29879]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[29879]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[29879]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[41444]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[41444]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[41444]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:43.300 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:42 smithi050 ceph-mon[33492]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:42 smithi050 ceph-mon[33492]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:30:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:42 smithi050 ceph-mon[33492]: osdmap e7: 1 total, 0 up, 1 in 2024-02-05T07:30:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:44.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:30:44.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:30:44.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[29879]: osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] boot 2024-02-05T07:30:44.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[29879]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T07:30:44.021 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:44.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:30:44.021 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:30:44.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[41444]: osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] boot 2024-02-05T07:30:44.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[41444]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T07:30:44.022 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:43 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:43 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:30:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:43 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:30:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:43 smithi050 ceph-mon[33492]: osd.0 [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] boot 2024-02-05T07:30:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:43 smithi050 ceph-mon[33492]: osdmap e8: 1 total, 1 up, 1 in 2024-02-05T07:30:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:43 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:30:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-05T07:30:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-05T07:30:45.243 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-02-05T07:30:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-02-05T07:30:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00863628 s, 1.2 GB/s 2024-02-05T07:30:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-02-05T07:30:45.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:45.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T07:30:45.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.243 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:45.244 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:45.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: osdmap e9: 1 total, 1 up, 1 in 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:30:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:30:45.841 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-02-05T07:30:46.086 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:30:46.205 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:45 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi043 2024-02-05T07:30:46.205 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:45 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi043 to 16069M 2024-02-05T07:30:46.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:45 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi043 2024-02-05T07:30:46.206 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:45 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi043 to 16069M 2024-02-05T07:30:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:45 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi043 2024-02-05T07:30:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:45 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi043 to 16069M 2024-02-05T07:30:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:46 smithi043 ceph-mon[29879]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:46 smithi043 ceph-mon[41444]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:46 smithi050 ceph-mon[33492]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:47.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 -- 172.21.15.43:0/1038874608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5698100060 msgr2=0x7f5698100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 --2- 172.21.15.43:0/1038874608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5698100060 0x7f5698100480 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f568c009990 tx=0x7f568c009ca0 comp rx=0 tx=0).stop 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 -- 172.21.15.43:0/1038874608 shutdown_connections 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 --2- 172.21.15.43:0/1038874608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56981039e0 0x7f569810a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 --2- 172.21.15.43:0/1038874608 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f5698100060 0x7f5698100480 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 --2- 172.21.15.43:0/1038874608 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698103040 0x7f56981034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 -- 172.21.15.43:0/1038874608 >> 172.21.15.43:0/1038874608 conn(0x7f56980fb9c0 msgr2=0x7f56980fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 -- 172.21.15.43:0/1038874608 shutdown_connections 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.835+0000 7f569ee97700 1 -- 172.21.15.43:0/1038874608 wait complete. 2024-02-05T07:30:47.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 Processor -- start 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 -- start start 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5698100060 0x7f5698197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698103040 0x7f5698197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56981039e0 0x7f569819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569819ed70 con 0x7f5698100060 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f569819eeb0 con 0x7f5698103040 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569ee97700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f569819f040 con 0x7f56981039e0 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569cc33700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5698100060 0x7f5698197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569d434700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56981039e0 0x7f569819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569cc33700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5698100060 0x7f5698197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38808/0 (socket says 172.21.15.43:38808) 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f569cc33700 1 -- 172.21.15.43:0/2833828740 learned_addr learned my addr 172.21.15.43:0/2833828740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:30:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.836+0000 7f5697fff700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698103040 0x7f5698197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569d434700 1 -- 172.21.15.43:0/2833828740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698103040 msgr2=0x7f5698197770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569d434700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698103040 0x7f5698197770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569d434700 1 -- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5698100060 msgr2=0x7f5698197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569d434700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5698100060 0x7f5698197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569d434700 1 -- 172.21.15.43:0/2833828740 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5690009c40 con 0x7f56981039e0 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569cc33700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5698100060 0x7f5698197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f5697fff700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698103040 0x7f5698197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569d434700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56981039e0 0x7f569819e7a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f569000c0c0 tx=0x7f569000c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f5695ffb700 1 -- 172.21.15.43:0/2833828740 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f569000ca80 con 0x7f56981039e0 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f5695ffb700 1 -- 172.21.15.43:0/2833828740 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f569000f040 con 0x7f56981039e0 2024-02-05T07:30:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568c0095f0 con 0x7f56981039e0 2024-02-05T07:30:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f5695ffb700 1 -- 172.21.15.43:0/2833828740 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5690004550 con 0x7f56981039e0 2024-02-05T07:30:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.837+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f569819f640 con 0x7f56981039e0 2024-02-05T07:30:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.838+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5698108a20 con 0x7f56981039e0 2024-02-05T07:30:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.838+0000 7f5695ffb700 1 -- 172.21.15.43:0/2833828740 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f56900046b0 con 0x7f56981039e0 2024-02-05T07:30:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.839+0000 7f5695ffb700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f568806d940 0x7f568806fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:30:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.839+0000 7f5695ffb700 1 -- 172.21.15.43:0/2833828740 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f569008bd90 con 0x7f56981039e0 2024-02-05T07:30:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.839+0000 7f569cc33700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f568806d940 0x7f568806fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:30:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.839+0000 7f569cc33700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f568806d940 0x7f568806fe00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f568c0058e0 tx=0x7f568c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:30:47.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:47.844+0000 7f5695ffb700 1 -- 172.21.15.43:0/2833828740 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56900568d0 con 0x7f56981039e0 2024-02-05T07:30:48.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:30:48.056+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f569810ce10 con 0x7f568806d940 2024-02-05T07:30:49.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[29879]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:30:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:30:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:49.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[41444]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:49.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:30:49.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:30:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:48 smithi050 ceph-mon[33492]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:30:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:30:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:50.051 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:49 smithi043 ceph-mon[29879]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:50.051 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:49 smithi043 ceph-mon[41444]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:49 smithi050 ceph-mon[33492]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:30:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[29879]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2560485964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]: dispatch 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]: dispatch 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]': finished 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[29879]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[41444]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2560485964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]: dispatch 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]: dispatch 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]': finished 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[41444]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T07:30:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:50 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:30:51.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:50 smithi050 ceph-mon[33492]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2560485964' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]: dispatch 2024-02-05T07:30:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]: dispatch 2024-02-05T07:30:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "903ff5b0-7d34-4899-8a55-50719f1a09da"}]': finished 2024-02-05T07:30:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:50 smithi050 ceph-mon[33492]: osdmap e10: 2 total, 1 up, 2 in 2024-02-05T07:30:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:50 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:30:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2607691297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:30:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2607691297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:30:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2607691297' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:30:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:52 smithi043 ceph-mon[29879]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:52 smithi043 ceph-mon[41444]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:52 smithi050 ceph-mon[33492]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:55.193 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:54 smithi043 ceph-mon[29879]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:55.193 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:54 smithi043 ceph-mon[41444]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:54 smithi050 ceph-mon[33492]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:57.203 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:56 smithi043 ceph-mon[29879]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:57.203 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:56 smithi043 ceph-mon[41444]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:57.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:56 smithi050 ceph-mon[33492]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:58.096 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:57 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T07:30:58.096 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:57 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:58.096 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:57 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T07:30:58.096 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:57 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:57 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-05T07:30:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:57 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:30:59.203 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:58 smithi043 ceph-mon[29879]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:59.204 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:30:58 smithi043 ceph-mon[29879]: Deploying daemon osd.1 on smithi043 2024-02-05T07:30:59.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:58 smithi043 ceph-mon[41444]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:59.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:30:58 smithi043 ceph-mon[41444]: Deploying daemon osd.1 on smithi043 2024-02-05T07:30:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:58 smithi050 ceph-mon[33492]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:30:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:30:58 smithi050 ceph-mon[33492]: Deploying daemon osd.1 on smithi043 2024-02-05T07:31:01.213 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:00 smithi043 ceph-mon[29879]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:01.213 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:00 smithi043 ceph-mon[41444]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:00 smithi050 ceph-mon[33492]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[29879]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[41444]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:02 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:02 smithi050 ceph-mon[33492]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:02 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:02 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:02 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:04 smithi043 ceph-mon[29879]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:05.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:04 smithi043 ceph-mon[41444]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:05.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:04 smithi050 ceph-mon[33492]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.458+0000 7f5695ffb700 1 -- 172.21.15.43:0/2833828740 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f569810ce10 con 0x7f568806d940 2024-02-05T07:31:06.460 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 1 on host 'smithi043' 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f568806d940 msgr2=0x7f568806fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f568806d940 0x7f568806fe00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f568c0058e0 tx=0x7f568c005960 comp rx=0 tx=0).stop 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56981039e0 msgr2=0x7f569819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56981039e0 0x7f569819e7a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f569000c0c0 tx=0x7f569000c3d0 comp rx=0 tx=0).stop 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 shutdown_connections 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f568806d940 0x7f568806fe00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f56981039e0 0x7f569819e7a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:06.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5698100060 0x7f5698197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 --2- 172.21.15.43:0/2833828740 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f5698103040 0x7f5698197770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.460+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 >> 172.21.15.43:0/2833828740 conn(0x7f56980fb9c0 msgr2=0x7f56980fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:31:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.461+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 shutdown_connections 2024-02-05T07:31:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:06.461+0000 7f569ee97700 1 -- 172.21.15.43:0/2833828740 wait complete. 2024-02-05T07:31:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:07.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:31:07.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:31:07.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:07.102 DEBUG:teuthology.orchestra.run.smithi043:osd.1> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.1.service 2024-02-05T07:31:07.104 INFO:tasks.cephadm:Deploying osd.2 on smithi043 with /dev/vg_nvme/lv_2... 2024-02-05T07:31:07.104 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-05T07:31:07.820 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:31:07.929 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[29879]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[29879]: from='osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[29879]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[29879]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[41444]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[41444]: from='osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[41444]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:07.930 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-mon[41444]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:07.931 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1[52148]: 2024-02-05T07:31:07.725+0000 7f7e1a5c3700 -1 osd.1 0 waiting for initial osdmap 2024-02-05T07:31:07.931 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:07 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1[52148]: 2024-02-05T07:31:07.727+0000 7f7e163bd700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:31:08.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:07 smithi050 ceph-mon[33492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-05T07:31:08.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:07 smithi050 ceph-mon[33492]: from='osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:08.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:07 smithi050 ceph-mon[33492]: osdmap e11: 2 total, 1 up, 2 in 2024-02-05T07:31:08.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:08.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:07 smithi050 ceph-mon[33492]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:08.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[29879]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:08.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[29879]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[29879]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[41444]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[41444]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[41444]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:08.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:08 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:08 smithi050 ceph-mon[33492]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:08 smithi050 ceph-mon[33492]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:31:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:08 smithi050 ceph-mon[33492]: osdmap e12: 2 total, 1 up, 2 in 2024-02-05T07:31:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:08 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:08 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[29879]: osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] boot 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[29879]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[41444]: osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] boot 2024-02-05T07:31:10.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[41444]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T07:31:10.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:09 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:09 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:31:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:09 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:31:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:09 smithi050 ceph-mon[33492]: osd.1 [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] boot 2024-02-05T07:31:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:09 smithi050 ceph-mon[33492]: osdmap e13: 2 total, 2 up, 2 in 2024-02-05T07:31:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:09 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:31:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T07:31:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:11.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:11.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:11.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:11.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: osdmap e14: 2 total, 2 up, 2 in 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:11.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-05T07:31:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-05T07:31:11.262 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-02-05T07:31:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-02-05T07:31:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00852629 s, 1.2 GB/s 2024-02-05T07:31:11.262 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 2024-02-05T07:31:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-02-05T07:31:11.861 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:11 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi043 2024-02-05T07:31:11.861 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:11 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi043 to 8034M 2024-02-05T07:31:11.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:11 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi043 2024-02-05T07:31:11.862 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:11 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi043 to 8034M 2024-02-05T07:31:11.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-02-05T07:31:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:11 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi043 2024-02-05T07:31:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:11 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi043 to 8034M 2024-02-05T07:31:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:31:13.042 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:12 smithi043 ceph-mon[29879]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:13.042 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:12 smithi043 ceph-mon[41444]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:13.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:12 smithi050 ceph-mon[33492]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 -- 172.21.15.43:0/3839818526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f10101e40 msgr2=0x7f7f101022a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 --2- 172.21.15.43:0/3839818526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f10101e40 0x7f7f101022a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80098f0 tx=0x7f7ef8009c00 comp rx=0 tx=0).stop 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 -- 172.21.15.43:0/3839818526 shutdown_connections 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 --2- 172.21.15.43:0/3839818526 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f101027e0 0x7f7f10109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 --2- 172.21.15.43:0/3839818526 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f10100c40 0x7f7f10101060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 --2- 172.21.15.43:0/3839818526 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f10101e40 0x7f7f101022a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 -- 172.21.15.43:0/3839818526 >> 172.21.15.43:0/3839818526 conn(0x7f7f100fc1e0 msgr2=0x7f7f100fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 -- 172.21.15.43:0/3839818526 shutdown_connections 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.760+0000 7f7f15442700 1 -- 172.21.15.43:0/3839818526 wait complete. 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 Processor -- start 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 -- start start 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f10100c40 0x7f7f10197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f10101e40 0x7f7f101977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f101027e0 0x7f7f1019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1019ee00 con 0x7f7f10101e40 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1019ef40 con 0x7f7f10100c40 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f15442700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1019f0d0 con 0x7f7f101027e0 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f101027e0 0x7f7f1019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f0f7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f101027e0 0x7f7f1019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:45636/0 (socket says 172.21.15.43:45636) 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f0f7fe700 1 -- 172.21.15.43:0/3570805666 learned_addr learned my addr 172.21.15.43:0/3570805666 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f0e7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f10101e40 0x7f7f101977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.761+0000 7f7f0effd700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f10100c40 0x7f7f10197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f0f7fe700 1 -- 172.21.15.43:0/3570805666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f10100c40 msgr2=0x7f7f10197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f0f7fe700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f10100c40 0x7f7f10197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f0f7fe700 1 -- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f10101e40 msgr2=0x7f7f101977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f0f7fe700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f10101e40 0x7f7f101977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f0f7fe700 1 -- 172.21.15.43:0/3570805666 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f080095f0 con 0x7f7f101027e0 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f0e7fc700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f10101e40 0x7f7f101977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f0f7fe700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f101027e0 0x7f7f1019e830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000ef60 tx=0x7f7f0000ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f07fff700 1 -- 172.21.15.43:0/3570805666 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f00018070 con 0x7f7f101027e0 2024-02-05T07:31:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef8009520 con 0x7f7f101027e0 2024-02-05T07:31:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f07fff700 1 -- 172.21.15.43:0/3570805666 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7f0000f040 con 0x7f7f101027e0 2024-02-05T07:31:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f07fff700 1 -- 172.21.15.43:0/3570805666 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000cb90 con 0x7f7f101027e0 2024-02-05T07:31:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.762+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1019f710 con 0x7f7f101027e0 2024-02-05T07:31:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.763+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f10108a20 con 0x7f7f101027e0 2024-02-05T07:31:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.764+0000 7f7f07fff700 1 -- 172.21.15.43:0/3570805666 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f7f00003a00 con 0x7f7f101027e0 2024-02-05T07:31:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.764+0000 7f7f07fff700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7efc06d940 0x7f7efc06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.764+0000 7f7f07fff700 1 -- 172.21.15.43:0/3570805666 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2199+0+0 (secure 0 0 0) 0x7f7f0008bd80 con 0x7f7f101027e0 2024-02-05T07:31:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.764+0000 7f7f0effd700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7efc06d940 0x7f7efc06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.764+0000 7f7f0effd700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7efc06d940 0x7f7efc06fe00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f10101ca0 tx=0x7f7f08009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:31:13.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.769+0000 7f7f07fff700 1 -- 172.21.15.43:0/3570805666 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f00056730 con 0x7f7f101027e0 2024-02-05T07:31:13.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:13.980+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f1010bca0 con 0x7f7efc06d940 2024-02-05T07:31:14.840 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[41444]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:14.840 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[41444]: from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:31:14.840 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:31:14.840 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:31:14.840 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:14.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[29879]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:14.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[29879]: from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:31:14.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:31:14.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:31:14.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:14 smithi050 ceph-mon[33492]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:14 smithi050 ceph-mon[33492]: from='client.24176 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:31:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:31:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:31:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:17.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[29879]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:17.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/153191850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]: dispatch 2024-02-05T07:31:17.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]: dispatch 2024-02-05T07:31:17.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]': finished 2024-02-05T07:31:17.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[29879]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T07:31:17.168 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:17.168 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[41444]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:17.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/153191850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]: dispatch 2024-02-05T07:31:17.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]: dispatch 2024-02-05T07:31:17.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]': finished 2024-02-05T07:31:17.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[41444]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T07:31:17.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:16 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:16 smithi050 ceph-mon[33492]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/153191850' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]: dispatch 2024-02-05T07:31:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]: dispatch 2024-02-05T07:31:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3527ab76-91de-487e-b5b1-6b5c1933b880"}]': finished 2024-02-05T07:31:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:16 smithi050 ceph-mon[33492]: osdmap e15: 3 total, 2 up, 3 in 2024-02-05T07:31:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:16 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:18.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3779282450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:31:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3779282450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:31:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3779282450' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:31:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:18 smithi043 ceph-mon[29879]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:18 smithi043 ceph-mon[41444]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:18 smithi050 ceph-mon[33492]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:21.165 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:20 smithi043 ceph-mon[29879]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:21.165 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:20 smithi043 ceph-mon[41444]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:21.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:20 smithi050 ceph-mon[33492]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:23.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:22 smithi043 ceph-mon[29879]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:23.293 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:22 smithi043 ceph-mon[41444]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:22 smithi050 ceph-mon[33492]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:24.204 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:23 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T07:31:24.204 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:23 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:24.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:23 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T07:31:24.204 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:23 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:23 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-05T07:31:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:23 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:25.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:24 smithi050 ceph-mon[33492]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:25.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:24 smithi050 ceph-mon[33492]: Deploying daemon osd.2 on smithi043 2024-02-05T07:31:25.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:24 smithi043 ceph-mon[29879]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:25.239 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:24 smithi043 ceph-mon[29879]: Deploying daemon osd.2 on smithi043 2024-02-05T07:31:25.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:24 smithi043 ceph-mon[41444]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:25.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:24 smithi043 ceph-mon[41444]: Deploying daemon osd.2 on smithi043 2024-02-05T07:31:27.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:26 smithi043 ceph-mon[29879]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:27.212 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:26 smithi043 ceph-mon[41444]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:26 smithi050 ceph-mon[33492]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:29.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[29879]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:29.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:29.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:29.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:29.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[41444]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:29.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:29.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:29.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:28 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:29.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:28 smithi050 ceph-mon[33492]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:29.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:28 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:29.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:28 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:29.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:28 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:31.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:30 smithi043 ceph-mon[29879]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:31.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:30 smithi043 ceph-mon[41444]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:30 smithi050 ceph-mon[33492]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:32.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.633+0000 7f7f07fff700 1 -- 172.21.15.43:0/3570805666 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7f1010bca0 con 0x7f7efc06d940 2024-02-05T07:31:32.636 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 2 on host 'smithi043' 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.635+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7efc06d940 msgr2=0x7f7efc06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.635+0000 7f7f15442700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7efc06d940 0x7f7efc06fe00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7f10101ca0 tx=0x7f7f08009310 comp rx=0 tx=0).stop 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.635+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f101027e0 msgr2=0x7f7f1019e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.635+0000 7f7f15442700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f101027e0 0x7f7f1019e830 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000ef60 tx=0x7f7f0000ef90 comp rx=0 tx=0).stop 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 shutdown_connections 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7efc06d940 0x7f7efc06fe00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7f101027e0 0x7f7f1019e830 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7f10100c40 0x7f7f10197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 --2- 172.21.15.43:0/3570805666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f10101e40 0x7f7f101977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 >> 172.21.15.43:0/3570805666 conn(0x7f7f100fc1e0 msgr2=0x7f7f100fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:31:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 shutdown_connections 2024-02-05T07:31:32.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:32.636+0000 7f7f15442700 1 -- 172.21.15.43:0/3570805666 wait complete. 2024-02-05T07:31:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[41444]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[41444]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T07:31:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[29879]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:33.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[29879]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T07:31:33.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:32 smithi050 ceph-mon[33492]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:32 smithi050 ceph-mon[33492]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-05T07:31:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:33.382 DEBUG:teuthology.orchestra.run.smithi043:osd.2> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.2.service 2024-02-05T07:31:33.385 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_1... 2024-02-05T07:31:33.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-05T07:31:34.078 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:31:34.199 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[29879]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T07:31:34.199 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[29879]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T07:31:34.199 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[29879]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:34.200 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:34.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[41444]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T07:31:34.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[41444]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T07:31:34.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[41444]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:34.200 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:33 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:33 smithi050 ceph-mon[33492]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-05T07:31:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:33 smithi050 ceph-mon[33492]: osdmap e16: 3 total, 2 up, 3 in 2024-02-05T07:31:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:33 smithi050 ceph-mon[33492]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:31:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:33 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:34.563 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2[58158]: 2024-02-05T07:31:34.197+0000 7f2077cf5700 -1 osd.2 0 waiting for initial osdmap 2024-02-05T07:31:34.563 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2[58158]: 2024-02-05T07:31:34.200+0000 7f206fae7700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:31:35.010 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[29879]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[29879]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[29879]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[29879]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[41444]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:35.011 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[41444]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:31:35.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[41444]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T07:31:35.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[41444]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' 2024-02-05T07:31:35.012 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:34 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:34 smithi050 ceph-mon[33492]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:34 smithi050 ceph-mon[33492]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:31:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:34 smithi050 ceph-mon[33492]: osdmap e17: 3 total, 2 up, 3 in 2024-02-05T07:31:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:34 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:34 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:34 smithi050 ceph-mon[33492]: from='osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121]' entity='osd.2' 2024-02-05T07:31:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:34 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:36.294 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[41444]: osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] boot 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[41444]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[29879]: osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] boot 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[29879]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T07:31:36.295 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:35 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:36.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:35 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:31:36.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:35 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:31:36.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:35 smithi050 ceph-mon[33492]: osd.2 [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] boot 2024-02-05T07:31:36.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:35 smithi050 ceph-mon[33492]: osdmap e18: 3 total, 3 up, 3 in 2024-02-05T07:31:36.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:35 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.275 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.276 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:37.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:36 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:37.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:37.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:37.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:37.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:36 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-05T07:31:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-05T07:31:37.497 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-02-05T07:31:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-02-05T07:31:37.497 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0085218 s, 1.2 GB/s 2024-02-05T07:31:37.498 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-02-05T07:31:38.161 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi043 2024-02-05T07:31:38.161 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi043 to 5356M 2024-02-05T07:31:38.161 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[29879]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T07:31:38.162 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:31:38.162 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi043 2024-02-05T07:31:38.162 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi043 to 5356M 2024-02-05T07:31:38.162 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[41444]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T07:31:38.162 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:37 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:31:38.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-02-05T07:31:38.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:37 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi043 2024-02-05T07:31:38.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:37 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi043 to 5356M 2024-02-05T07:31:38.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:37 smithi050 ceph-mon[33492]: osdmap e19: 3 total, 3 up, 3 in 2024-02-05T07:31:38.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:37 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:31:38.441 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:31:39.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[29879]: pgmap v80: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:39.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T07:31:39.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[29879]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T07:31:39.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:31:39.245 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[41444]: pgmap v80: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:39.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T07:31:39.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[41444]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T07:31:39.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:38 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:31:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:38 smithi050 ceph-mon[33492]: pgmap v80: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:38 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-05T07:31:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:38 smithi050 ceph-mon[33492]: osdmap e20: 3 total, 3 up, 3 in 2024-02-05T07:31:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:38 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:31:39.813 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62646]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:31:39.813 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62646]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:39.813 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62646]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:39.813 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62646]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:40.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.129+0000 7f2337bd1700 1 -- 172.21.15.43:0/3389126714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 msgr2=0x7f23301022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.129+0000 7f2337bd1700 1 --2- 172.21.15.43:0/3389126714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 0x7f23301022b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2320009990 tx=0x7f2320009ca0 comp rx=0 tx=0).stop 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.129+0000 7f2337bd1700 1 -- 172.21.15.43:0/3389126714 shutdown_connections 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.129+0000 7f2337bd1700 1 --2- 172.21.15.43:0/3389126714 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 0x7f23301090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.129+0000 7f2337bd1700 1 --2- 172.21.15.43:0/3389126714 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2330100c50 0x7f2330101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.129+0000 7f2337bd1700 1 --2- 172.21.15.43:0/3389126714 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 0x7f23301022b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.129+0000 7f2337bd1700 1 -- 172.21.15.43:0/3389126714 >> 172.21.15.43:0/3389126714 conn(0x7f23300fc1f0 msgr2=0x7f23300fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.130+0000 7f2337bd1700 1 -- 172.21.15.43:0/3389126714 shutdown_connections 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.130+0000 7f2337bd1700 1 -- 172.21.15.43:0/3389126714 wait complete. 2024-02-05T07:31:40.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.130+0000 7f2337bd1700 1 Processor -- start 2024-02-05T07:31:40.133 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:39 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:31:40.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:39 smithi043 ceph-mon[29879]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T07:31:40.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:39 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:40.134 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:39 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:31:40.134 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:39 smithi043 ceph-mon[41444]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T07:31:40.134 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:39 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:40.134 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62668]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:31:40.134 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62668]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:40.134 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62668]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:40.134 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:31:39 smithi043 sudo[62668]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:40.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.130+0000 7f2337bd1700 1 -- start start 2024-02-05T07:31:40.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f2337bd1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2330100c50 0x7f23301961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:40.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f2337bd1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 0x7f23301966f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:40.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f2337bd1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 0x7f233019d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:40.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f2337bd1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233019dd40 con 0x7f23301027f0 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f2337bd1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23300699d0 con 0x7f2330100c50 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f2337bd1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2330069b90 con 0x7f2330101e50 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 0x7f233019d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 0x7f233019d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48550/0 (socket says 172.21.15.43:48550) 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 -- 172.21.15.43:0/2390771465 learned_addr learned my addr 172.21.15.43:0/2390771465 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233516c700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 0x7f23301966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233596d700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2330100c50 0x7f23301961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 -- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 msgr2=0x7f23301966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 0x7f23301966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 -- 172.21.15.43:0/2390771465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2330100c50 msgr2=0x7f23301961b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2330100c50 0x7f23301961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 -- 172.21.15.43:0/2390771465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2328009c40 con 0x7f23301027f0 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f233616e700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 0x7f233019d770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f232800c370 tx=0x7f232800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:31:40.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.131+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328017070 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.132+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23200095f0 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.132+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233006a180 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.132+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f232800f040 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.132+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800e050 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.133+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2314005150 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.134+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 14) v1 ==== 91350+0+0 (secure 0 0 0) 0x7f2328003a00 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.134+0000 7f2326ffd700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f231c06d940 0x7f231c06fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.134+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f232808bdc0 con 0x7f23301027f0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.134+0000 7f233596d700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f231c06d940 0x7f231c06fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:31:40.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.135+0000 7f233596d700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f231c06d940 0x7f231c06fe00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f232c005950 tx=0x7f232c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:31:40.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.140+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2328056440 con 0x7f23301027f0 2024-02-05T07:31:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:39 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:31:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:39 smithi050 ceph-mon[33492]: osdmap e21: 3 total, 3 up, 3 in 2024-02-05T07:31:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:39 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:40.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.354+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2314000bf0 con 0x7f231c06d940 2024-02-05T07:31:40.453 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62698]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:31:40.454 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62698]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:40.454 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62698]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:40.454 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62698]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:40.713 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62768]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:31:40.713 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62768]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:40.713 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62768]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:40.713 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62768]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:40.902+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f2328056810 con 0x7f23301027f0 2024-02-05T07:31:40.987 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62771]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:31:40.987 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62771]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:40.987 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62771]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:40.987 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62771]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:40.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[29879]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:40.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[29879]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T07:31:40.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:31:40.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:31:40.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:40.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[29879]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-05T07:31:40.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[41444]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:40.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[41444]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T07:31:40.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:31:40.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:31:40.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:40.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:40 smithi043 ceph-mon[41444]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-05T07:31:41.258 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62830]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:31:41.258 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:41.258 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62830]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:41.258 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:31:40 smithi043 sudo[62830]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:40 smithi050 ceph-mon[33492]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:40 smithi050 ceph-mon[33492]: osdmap e22: 3 total, 3 up, 3 in 2024-02-05T07:31:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:31:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:31:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:40 smithi050 ceph-mon[33492]: mgrmap e15: y(active, since 2m), standbys: x 2024-02-05T07:31:41.525 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62853]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:31:41.571 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62853]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:41.571 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62853]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:41.571 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62853]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62874]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:31:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62874]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62874]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:41 smithi043 sudo[62874]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:42.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:41 smithi050 sudo[36334]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:31:42.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:41 smithi050 sudo[36334]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:42.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:41 smithi050 sudo[36334]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:42.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:41 smithi050 sudo[36334]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:42.302 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:31:42.302 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:42.302 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:42.303 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:42.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:42.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:42.304 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 sudo[36337]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:31:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 sudo[36337]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 sudo[36337]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 sudo[36337]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:31:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:42.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:42.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:42.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:42.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:42.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:42.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:42.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62926]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-05T07:31:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62926]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62926]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62926]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62939]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-05T07:31:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62939]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-05T07:31:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62939]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-05T07:31:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:42 smithi043 sudo[62939]: pam_unix(sudo:session): session closed for user root 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:43.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:43 smithi050 ceph-mon[33492]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:43.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:43.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:43.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:43.546 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[41444]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-05T07:31:43.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:31:43.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:31:43.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:31:43.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:43 smithi043 ceph-mon[29879]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-05T07:31:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[41444]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1850610706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a1431fa-3b8a-4aec-aaf4-58fa8ff28175"}]: dispatch 2024-02-05T07:31:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1850610706' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a1431fa-3b8a-4aec-aaf4-58fa8ff28175"}]': finished 2024-02-05T07:31:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[41444]: osdmap e23: 4 total, 3 up, 4 in 2024-02-05T07:31:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:31:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3015892306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:31:44.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[29879]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:44.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1850610706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a1431fa-3b8a-4aec-aaf4-58fa8ff28175"}]: dispatch 2024-02-05T07:31:44.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1850610706' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a1431fa-3b8a-4aec-aaf4-58fa8ff28175"}]': finished 2024-02-05T07:31:44.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[29879]: osdmap e23: 4 total, 3 up, 4 in 2024-02-05T07:31:44.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:31:44.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3015892306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:31:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:44 smithi050 ceph-mon[33492]: pgmap v86: 1 pgs: 1 active+clean; 449 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1850610706' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a1431fa-3b8a-4aec-aaf4-58fa8ff28175"}]: dispatch 2024-02-05T07:31:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1850610706' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a1431fa-3b8a-4aec-aaf4-58fa8ff28175"}]': finished 2024-02-05T07:31:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:44 smithi050 ceph-mon[33492]: osdmap e23: 4 total, 3 up, 4 in 2024-02-05T07:31:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:44 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:31:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3015892306' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:31:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:46 smithi043 ceph-mon[29879]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 5.3 KiB/s rd, 256 KiB/s wr, 17 op/s 2024-02-05T07:31:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:46 smithi043 ceph-mon[41444]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 5.3 KiB/s rd, 256 KiB/s wr, 17 op/s 2024-02-05T07:31:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:46 smithi050 ceph-mon[33492]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 5.3 KiB/s rd, 256 KiB/s wr, 17 op/s 2024-02-05T07:31:49.052 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:48 smithi043 ceph-mon[41444]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 184 KiB/s wr, 12 op/s 2024-02-05T07:31:49.052 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:48 smithi043 ceph-mon[29879]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 184 KiB/s wr, 12 op/s 2024-02-05T07:31:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:48 smithi050 ceph-mon[33492]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.8 KiB/s rd, 184 KiB/s wr, 12 op/s 2024-02-05T07:31:51.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[41444]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 143 KiB/s wr, 9 op/s 2024-02-05T07:31:51.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-05T07:31:51.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:51.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[41444]: Deploying daemon osd.3 on smithi043 2024-02-05T07:31:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[29879]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 143 KiB/s wr, 9 op/s 2024-02-05T07:31:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-05T07:31:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:50 smithi043 ceph-mon[29879]: Deploying daemon osd.3 on smithi043 2024-02-05T07:31:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:50 smithi050 ceph-mon[33492]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 143 KiB/s wr, 9 op/s 2024-02-05T07:31:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:50 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-05T07:31:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:50 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:50 smithi050 ceph-mon[33492]: Deploying daemon osd.3 on smithi043 2024-02-05T07:31:53.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:52 smithi043 ceph-mon[29879]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-05T07:31:53.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:52 smithi043 ceph-mon[41444]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-05T07:31:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:52 smithi050 ceph-mon[33492]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-05T07:31:55.202 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:54 smithi043 ceph-mon[29879]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-05T07:31:55.202 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:54 smithi043 ceph-mon[41444]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-05T07:31:55.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:54 smithi050 ceph-mon[33492]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.7 KiB/s rd, 131 KiB/s wr, 8 op/s 2024-02-05T07:31:56.093 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:55 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:56.094 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:55 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:56.094 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:55 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:56.094 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:55 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:56.094 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:55 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:56.094 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:55 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:56.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:55 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:31:56.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:55 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:56.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:55 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:56 smithi043 ceph-mon[29879]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 110 KiB/s wr, 7 op/s 2024-02-05T07:31:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:56 smithi043 ceph-mon[41444]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 110 KiB/s wr, 7 op/s 2024-02-05T07:31:57.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:56 smithi050 ceph-mon[33492]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.3 KiB/s rd, 110 KiB/s wr, 7 op/s 2024-02-05T07:31:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:59.146 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: from='osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[29879]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.147 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: from='osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:31:59.148 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:31:58 smithi043 ceph-mon[41444]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:31:59.216 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-02-05T07:31:59.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.214+0000 7f2326ffd700 1 -- 172.21.15.43:0/2390771465 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2314000bf0 con 0x7f231c06d940 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.216+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f231c06d940 msgr2=0x7f231c06fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.216+0000 7f2337bd1700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f231c06d940 0x7f231c06fe00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f232c005950 tx=0x7f232c0058e0 comp rx=0 tx=0).stop 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.216+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 msgr2=0x7f233019d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.216+0000 7f2337bd1700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 0x7f233019d770 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f232800c370 tx=0x7f232800c680 comp rx=0 tx=0).stop 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 shutdown_connections 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f231c06d940 0x7f231c06fe00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23301027f0 0x7f233019d770 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2330100c50 0x7f23301961b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:59.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 --2- 172.21.15.43:0/2390771465 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2330101e50 0x7f23301966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:31:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 >> 172.21.15.43:0/2390771465 conn(0x7f23300fc1f0 msgr2=0x7f23300fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:31:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 shutdown_connections 2024-02-05T07:31:59.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:31:59.217+0000 7f2337bd1700 1 -- 172.21.15.43:0/2390771465 wait complete. 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: from='osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:31:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:31:58 smithi050 ceph-mon[33492]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-05T07:31:59.909 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.3.service 2024-02-05T07:31:59.911 INFO:tasks.cephadm:Deploying osd.4 on smithi050 with /dev/vg_nvme/lv_4... 2024-02-05T07:31:59.912 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-05T07:32:00.161 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: osdmap e24: 4 total, 3 up, 4 in 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:00.283 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: from='osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:32:00.284 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:00 smithi050 ceph-mon[33492]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:32:00.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:32:00.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: osdmap e24: 4 total, 3 up, 4 in 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: from='osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[41444]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:00.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:00.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-05T07:32:00.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: osdmap e24: 4 total, 3 up, 4 in 2024-02-05T07:32:00.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:00.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: from='osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:32:00.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-mon[29879]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-02-05T07:32:00.813 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3[64796]: 2024-02-05T07:32:00.682+0000 7f498a07d700 -1 osd.3 0 waiting for initial osdmap 2024-02-05T07:32:00.813 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 07:32:00 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3[64796]: 2024-02-05T07:32:00.686+0000 7f4981e6f700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:32:01.513 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[29879]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[29879]: osdmap e25: 4 total, 3 up, 4 in 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[29879]: from='osd.3 ' entity='osd.3' 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[41444]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[41444]: osdmap e25: 4 total, 3 up, 4 in 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:01.514 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:01 smithi043 ceph-mon[41444]: from='osd.3 ' entity='osd.3' 2024-02-05T07:32:01.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:01 smithi050 ceph-mon[33492]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-02-05T07:32:01.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:01 smithi050 ceph-mon[33492]: osdmap e25: 4 total, 3 up, 4 in 2024-02-05T07:32:01.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:01 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:01.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:01 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:01.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:01 smithi050 ceph-mon[33492]: from='osd.3 ' entity='osd.3' 2024-02-05T07:32:02.123 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-05T07:32:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-05T07:32:02.124 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-05T07:32:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-05T07:32:02.124 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00887236 s, 1.2 GB/s 2024-02-05T07:32:02.124 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[29879]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[29879]: osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] boot 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[29879]: osdmap e26: 4 total, 4 up, 4 in 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:32:02.682 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[41444]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:32:02.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:02.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[41444]: osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] boot 2024-02-05T07:32:02.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[41444]: osdmap e26: 4 total, 4 up, 4 in 2024-02-05T07:32:02.683 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:02 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:02.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:02 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:32:02.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:02 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:32:02.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:02 smithi050 ceph-mon[33492]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-05T07:32:02.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:02 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:02.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:02 smithi050 ceph-mon[33492]: osd.3 [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] boot 2024-02-05T07:32:02.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:02 smithi050 ceph-mon[33492]: osdmap e26: 4 total, 4 up, 4 in 2024-02-05T07:32:02.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:02 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:32:02.784 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_4 2024-02-05T07:32:03.028 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:32:04.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:04.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi043 2024-02-05T07:32:04.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi043 to 4017M 2024-02-05T07:32:04.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:04.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:04.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:04 smithi050 ceph-mon[33492]: osdmap e27: 4 total, 4 up, 4 in 2024-02-05T07:32:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi043 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi043 to 4017M 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[29879]: osdmap e27: 4 total, 4 up, 4 in 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi043 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi043 to 4017M 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:04 smithi043 ceph-mon[41444]: osdmap e27: 4 total, 4 up, 4 in 2024-02-05T07:32:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/975191309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8101d50 msgr2=0x7f1ff8102170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/975191309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8101d50 0x7f1ff8102170 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80099c0 tx=0x7f1fe8009cd0 comp rx=0 tx=0).stop 2024-02-05T07:32:04.644 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/975191309 shutdown_connections 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/975191309 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ff81038f0 0x7f1ff810a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/975191309 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8101d50 0x7f1ff8102170 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/975191309 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff8102f50 0x7f1ff81033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/975191309 >> 172.21.15.50:0/975191309 conn(0x7f1ff80fd330 msgr2=0x7f1ff80ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/975191309 shutdown_connections 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.640+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/975191309 wait complete. 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 Processor -- start 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 -- start start 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff8101d50 0x7f1ff8075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8102f50 0x7f1ff8074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ff81038f0 0x7f1ff8074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:04.645 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8074dd0 con 0x7f1ff8101d50 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff8074f40 con 0x7f1ff8102f50 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ffc9c6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff81a13d0 con 0x7f1ff81038f0 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ff5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8102f50 0x7f1ff8074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ff5d9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8102f50 0x7f1ff8074110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54178/0 (socket says 172.21.15.50:54178) 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ff6d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ff81038f0 0x7f1ff8074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ff659c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff8101d50 0x7f1ff8075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.641+0000 7f1ff5d9b700 1 -- 172.21.15.50:0/2827408541 learned_addr learned my addr 172.21.15.50:0/2827408541 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff659c700 1 -- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ff81038f0 msgr2=0x7f1ff8074650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff659c700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ff81038f0 0x7f1ff8074650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff659c700 1 -- 172.21.15.50:0/2827408541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8102f50 msgr2=0x7f1ff8074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff659c700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8102f50 0x7f1ff8074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:04.646 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff659c700 1 -- 172.21.15.50:0/2827408541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80095f0 con 0x7f1ff8101d50 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff6d9d700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ff81038f0 0x7f1ff8074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff5d9b700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8102f50 0x7f1ff8074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ff659c700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff8101d50 0x7f1ff8075ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009990 tx=0x7f1fe800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1fe77fe700 1 -- 172.21.15.50:0/2827408541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe801d070 con 0x7f1ff8101d50 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1fe77fe700 1 -- 172.21.15.50:0/2827408541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1fe800fe90 con 0x7f1ff8101d50 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff81a1650 con 0x7f1ff8101d50 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1fe77fe700 1 -- 172.21.15.50:0/2827408541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8022d90 con 0x7f1ff8101d50 2024-02-05T07:32:04.647 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.642+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff81a1ae0 con 0x7f1ff8101d50 2024-02-05T07:32:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.643+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff8108930 con 0x7f1ff8101d50 2024-02-05T07:32:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.644+0000 7f1fe77fe700 1 -- 172.21.15.50:0/2827408541 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f1fe80225e0 con 0x7f1ff8101d50 2024-02-05T07:32:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.644+0000 7f1fe77fe700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1fe006d9d0 0x7f1fe006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.645+0000 7f1fe77fe700 1 -- 172.21.15.50:0/2827408541 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3449+0+0 (secure 0 0 0) 0x7f1fe808dd50 con 0x7f1ff8101d50 2024-02-05T07:32:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.645+0000 7f1ff5d9b700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1fe006d9d0 0x7f1fe006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:04.648 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.645+0000 7f1ff5d9b700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1fe006d9d0 0x7f1fe006fe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009ea0 tx=0x7f1fec009290 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:32:04.650 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.649+0000 7f1fe77fe700 1 -- 172.21.15.50:0/2827408541 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1fe80582a0 con 0x7f1ff8101d50 2024-02-05T07:32:04.862 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:04.861+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ff8062160 con 0x7f1fe006d9d0 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[29879]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[41444]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:05.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:05 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:05.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:05 smithi050 ceph-mon[33492]: from='client.14331 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:05.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:05.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:05.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:05 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:07.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:06 smithi043 ceph-mon[29879]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:06 smithi043 ceph-mon[41444]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:06 smithi050 ceph-mon[33492]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/3045778629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]: dispatch 2024-02-05T07:32:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]: dispatch 2024-02-05T07:32:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]': finished 2024-02-05T07:32:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[29879]: osdmap e28: 5 total, 4 up, 5 in 2024-02-05T07:32:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:08.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/3045778629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]: dispatch 2024-02-05T07:32:08.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]: dispatch 2024-02-05T07:32:08.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]': finished 2024-02-05T07:32:08.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[41444]: osdmap e28: 5 total, 4 up, 5 in 2024-02-05T07:32:08.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/3045778629' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]: dispatch 2024-02-05T07:32:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]: dispatch 2024-02-05T07:32:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "79c3dcf9-499c-4223-a1ce-b8cace5dc16a"}]': finished 2024-02-05T07:32:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:07 smithi050 ceph-mon[33492]: osdmap e28: 5 total, 4 up, 5 in 2024-02-05T07:32:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:08 smithi043 ceph-mon[29879]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/3915595118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:32:09.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:08 smithi043 ceph-mon[41444]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:09.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/3915595118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:32:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:08 smithi050 ceph-mon[33492]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/3915595118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:32:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:10 smithi050 ceph-mon[33492]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:10 smithi043 ceph-mon[29879]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:11.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:10 smithi043 ceph-mon[41444]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:13.186 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:12 smithi050 ceph-mon[33492]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:12 smithi043 ceph-mon[29879]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:13.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:12 smithi043 ceph-mon[41444]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:15.203 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:14 smithi050 ceph-mon[33492]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:15.203 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-05T07:32:15.203 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:14 smithi043 ceph-mon[29879]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-05T07:32:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:14 smithi043 ceph-mon[41444]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-05T07:32:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:16.077 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:15 smithi050 ceph-mon[33492]: Deploying daemon osd.4 on smithi050 2024-02-05T07:32:16.183 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:15 smithi043 ceph-mon[29879]: Deploying daemon osd.4 on smithi050 2024-02-05T07:32:16.183 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:15 smithi043 ceph-mon[41444]: Deploying daemon osd.4 on smithi050 2024-02-05T07:32:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:16 smithi043 ceph-mon[29879]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:16 smithi043 ceph-mon[41444]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:17.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:16 smithi050 ceph-mon[33492]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:19.180 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:18 smithi050 ceph-mon[33492]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:18 smithi043 ceph-mon[29879]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:18 smithi043 ceph-mon[41444]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[29879]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[41444]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:21.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:20 smithi050 ceph-mon[33492]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:23.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[29879]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:23.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:23.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:23.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[41444]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:23.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:23.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:23.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:22 smithi050 ceph-mon[33492]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:23.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:23.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:23.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.677 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.676+0000 7f1fe77fe700 1 -- 172.21.15.50:0/2827408541 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1ff8062160 con 0x7f1fe006d9d0 2024-02-05T07:32:23.677 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 4 on host 'smithi050' 2024-02-05T07:32:23.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1fe006d9d0 msgr2=0x7f1fe006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:23.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1fe006d9d0 0x7f1fe006fe90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009ea0 tx=0x7f1fec009290 comp rx=0 tx=0).stop 2024-02-05T07:32:23.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff8101d50 msgr2=0x7f1ff8075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:23.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff8101d50 0x7f1ff8075ac0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009990 tx=0x7f1fe800f690 comp rx=0 tx=0).stop 2024-02-05T07:32:23.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 shutdown_connections 2024-02-05T07:32:23.679 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1fe006d9d0 0x7f1fe006fe90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:23.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1ff81038f0 0x7f1ff8074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:23.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff8101d50 0x7f1ff8075ac0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:23.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 --2- 172.21.15.50:0/2827408541 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1ff8102f50 0x7f1ff8074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:23.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 >> 172.21.15.50:0/2827408541 conn(0x7f1ff80fd330 msgr2=0x7f1ff80fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:32:23.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 shutdown_connections 2024-02-05T07:32:23.680 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:23.678+0000 7f1ffc9c6700 1 -- 172.21.15.50:0/2827408541 wait complete. 2024-02-05T07:32:23.986 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:23 smithi050 ceph-mon[33492]: from='osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:32:23.986 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:23 smithi050 ceph-mon[33492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:32:23.986 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:23 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.986 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:23 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:23.986 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:23 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:24.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[29879]: from='osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:32:24.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[29879]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:32:24.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:24.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:24.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:24.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[41444]: from='osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:32:24.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[41444]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-05T07:32:24.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:24.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:24.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:23 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:24.271 DEBUG:teuthology.orchestra.run.smithi050:osd.4> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.4.service 2024-02-05T07:32:24.273 INFO:tasks.cephadm:Deploying osd.5 on smithi050 with /dev/vg_nvme/lv_3... 2024-02-05T07:32:24.273 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-05T07:32:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:24 smithi050 ceph-mon[33492]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:24 smithi050 ceph-mon[33492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-05T07:32:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:24 smithi050 ceph-mon[33492]: from='osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:24 smithi050 ceph-mon[33492]: osdmap e29: 5 total, 4 up, 5 in 2024-02-05T07:32:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:24 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:24 smithi050 ceph-mon[33492]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:25.051 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:32:25.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[29879]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:25.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[29879]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-05T07:32:25.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[29879]: from='osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:25.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[29879]: osdmap e29: 5 total, 4 up, 5 in 2024-02-05T07:32:25.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:25.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[29879]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:25.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[41444]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:25.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[41444]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-05T07:32:25.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[41444]: from='osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:25.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[41444]: osdmap e29: 5 total, 4 up, 5 in 2024-02-05T07:32:25.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:25.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:24 smithi043 ceph-mon[41444]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:25.173 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T07:32:25.027+0000 7fc2f344c700 -1 osd.4 0 waiting for initial osdmap 2024-02-05T07:32:25.173 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T07:32:25.031+0000 7fc2eaa3d700 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:32:26.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[29879]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:32:26.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[29879]: osdmap e30: 5 total, 4 up, 5 in 2024-02-05T07:32:26.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[29879]: osdmap e31: 5 total, 4 up, 5 in 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[29879]: from='osd.4 ' entity='osd.4' 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[41444]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[41444]: osdmap e30: 5 total, 4 up, 5 in 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[41444]: osdmap e31: 5 total, 4 up, 5 in 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:25 smithi043 ceph-mon[41444]: from='osd.4 ' entity='osd.4' 2024-02-05T07:32:26.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-mon[33492]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:32:26.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-mon[33492]: osdmap e30: 5 total, 4 up, 5 in 2024-02-05T07:32:26.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-mon[33492]: osdmap e31: 5 total, 4 up, 5 in 2024-02-05T07:32:26.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:26.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:25 smithi050 ceph-mon[33492]: from='osd.4 ' entity='osd.4' 2024-02-05T07:32:27.006 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:26 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:32:27.006 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:26 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:32:27.006 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:26 smithi050 ceph-mon[33492]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:27.006 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:26 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:27.006 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:26 smithi050 ceph-mon[33492]: osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] boot 2024-02-05T07:32:27.006 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:26 smithi050 ceph-mon[33492]: osdmap e32: 5 total, 5 up, 5 in 2024-02-05T07:32:27.006 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:26 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:27.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:32:27.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:32:27.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[29879]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:27.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:27.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[29879]: osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] boot 2024-02-05T07:32:27.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[29879]: osdmap e32: 5 total, 5 up, 5 in 2024-02-05T07:32:27.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:27.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:32:27.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:32:27.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[41444]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-02-05T07:32:27.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:27.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[41444]: osd.4 [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] boot 2024-02-05T07:32:27.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[41444]: osdmap e32: 5 total, 5 up, 5 in 2024-02-05T07:32:27.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:26 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:32:27.862 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-05T07:32:27.862 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-05T07:32:27.862 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-05T07:32:27.862 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-05T07:32:27.862 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00984575 s, 1.1 GB/s 2024-02-05T07:32:27.863 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-05T07:32:28.213 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:27 smithi050 ceph-mon[33492]: osdmap e33: 5 total, 5 up, 5 in 2024-02-05T07:32:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:27 smithi043 ceph-mon[29879]: osdmap e33: 5 total, 5 up, 5 in 2024-02-05T07:32:28.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:27 smithi043 ceph-mon[41444]: osdmap e33: 5 total, 5 up, 5 in 2024-02-05T07:32:28.490 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_3 2024-02-05T07:32:29.062 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:32:29.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:28 smithi050 ceph-mon[33492]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:29.184 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:28 smithi050 ceph-mon[33492]: osdmap e34: 5 total, 5 up, 5 in 2024-02-05T07:32:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:28 smithi043 ceph-mon[29879]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:28 smithi043 ceph-mon[29879]: osdmap e34: 5 total, 5 up, 5 in 2024-02-05T07:32:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:28 smithi043 ceph-mon[41444]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:28 smithi043 ceph-mon[41444]: osdmap e34: 5 total, 5 up, 5 in 2024-02-05T07:32:29.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: osdmap e35: 5 total, 5 up, 5 in 2024-02-05T07:32:29.928 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi050 2024-02-05T07:32:29.928 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: osdmap e35: 5 total, 5 up, 5 in 2024-02-05T07:32:30.181 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: osdmap e35: 5 total, 5 up, 5 in 2024-02-05T07:32:30.181 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi050 2024-02-05T07:32:30.181 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi050 to 17093M 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.182 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:29 smithi050 ceph-mon[33492]: osdmap e36: 5 total, 5 up, 5 in 2024-02-05T07:32:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi050 to 17093M 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[29879]: osdmap e36: 5 total, 5 up, 5 in 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi050 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi050 to 17093M 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:30.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:30.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:30.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:29 smithi043 ceph-mon[41444]: osdmap e36: 5 total, 5 up, 5 in 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 -- 172.21.15.50:0/2673968366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 msgr2=0x7f8864109eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 --2- 172.21.15.50:0/2673968366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864109eb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f885c009990 tx=0x7f885c009ca0 comp rx=0 tx=0).stop 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 -- 172.21.15.50:0/2673968366 shutdown_connections 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 --2- 172.21.15.50:0/2673968366 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864109eb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 --2- 172.21.15.50:0/2673968366 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88641027e0 0x7f8864104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 --2- 172.21.15.50:0/2673968366 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8864105110 0x7f8864107500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 -- 172.21.15.50:0/2673968366 >> 172.21.15.50:0/2673968366 conn(0x7f88640fc1f0 msgr2=0x7f88640fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:32:30.694 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 -- 172.21.15.50:0/2673968366 shutdown_connections 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.690+0000 7f886b2a0700 1 -- 172.21.15.50:0/2673968366 wait complete. 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 Processor -- start 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 -- start start 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88641027e0 0x7f88641020f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8864105110 0x7f8864100790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8864101470 con 0x7f8864105110 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88641015e0 con 0x7f8864107a40 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886b2a0700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88641a1250 con 0x7f88641027e0 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886903c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88641027e0 0x7f88641020f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886883b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8864105110 0x7f8864100790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:30.695 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:60510/0 (socket says 172.21.15.50:60510) 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 -- 172.21.15.50:0/3143788431 learned_addr learned my addr 172.21.15.50:0/3143788431 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 -- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88641027e0 msgr2=0x7f88641020f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88641027e0 0x7f88641020f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 -- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8864105110 msgr2=0x7f8864100790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8864105110 0x7f8864100790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.691+0000 7f886983d700 1 -- 172.21.15.50:0/3143788431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0095f0 con 0x7f8864107a40 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.692+0000 7f886983d700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864100cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f885c0036f0 tx=0x7f885c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.692+0000 7f88627fc700 1 -- 172.21.15.50:0/3143788431 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c01d070 con 0x7f8864107a40 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.692+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88641a1480 con 0x7f8864107a40 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.692+0000 7f88627fc700 1 -- 172.21.15.50:0/3143788431 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f885c00bd70 con 0x7f8864107a40 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.692+0000 7f88627fc700 1 -- 172.21.15.50:0/3143788431 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c022ba0 con 0x7f8864107a40 2024-02-05T07:32:30.696 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.692+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88641a18f0 con 0x7f8864107a40 2024-02-05T07:32:30.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.693+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88640fdd70 con 0x7f8864107a40 2024-02-05T07:32:30.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.693+0000 7f88627fc700 1 -- 172.21.15.50:0/3143788431 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f885c00fa90 con 0x7f8864107a40 2024-02-05T07:32:30.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.694+0000 7f88627fc700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f884c06d9d0 0x7f884c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:30.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.694+0000 7f88627fc700 1 -- 172.21.15.50:0/3143788431 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 3991+0+0 (secure 0 0 0) 0x7f885c08df70 con 0x7f8864107a40 2024-02-05T07:32:30.697 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.694+0000 7f886903c700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f884c06d9d0 0x7f884c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:30.698 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.694+0000 7f886903c700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f884c06d9d0 0x7f884c06fe90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8854009550 tx=0x7f88540091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:32:30.700 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.699+0000 7f88627fc700 1 -- 172.21.15.50:0/3143788431 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f885c0c1d20 con 0x7f8864107a40 2024-02-05T07:32:30.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:30.911+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8864062160 con 0x7f884c06d9d0 2024-02-05T07:32:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[29879]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[41444]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:30 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:31.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:30 smithi050 ceph-mon[33492]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:31.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:30 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:31.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:30 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:30 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:32.223 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:31 smithi050 ceph-mon[33492]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:31 smithi043 ceph-mon[29879]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:32.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:31 smithi043 ceph-mon[41444]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:32 smithi043 ceph-mon[29879]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-02-05T07:32:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:32 smithi043 ceph-mon[41444]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-02-05T07:32:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:32 smithi050 ceph-mon[33492]: pgmap v124: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 139 KiB/s, 0 objects/s recovering 2024-02-05T07:32:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/1149282901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]: dispatch 2024-02-05T07:32:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]: dispatch 2024-02-05T07:32:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]': finished 2024-02-05T07:32:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[29879]: osdmap e37: 6 total, 5 up, 6 in 2024-02-05T07:32:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:34.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/1149282901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]: dispatch 2024-02-05T07:32:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]: dispatch 2024-02-05T07:32:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]': finished 2024-02-05T07:32:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[41444]: osdmap e37: 6 total, 5 up, 6 in 2024-02-05T07:32:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:34 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:34.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/1149282901' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]: dispatch 2024-02-05T07:32:34.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]: dispatch 2024-02-05T07:32:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28fb0eeb-fce2-43e1-b5ee-e298db27ea98"}]': finished 2024-02-05T07:32:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:34 smithi050 ceph-mon[33492]: osdmap e37: 6 total, 5 up, 6 in 2024-02-05T07:32:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:34 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:35.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:35 smithi043 ceph-mon[29879]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:32:35.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/1267346570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:32:35.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:35 smithi043 ceph-mon[41444]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:32:35.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/1267346570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:32:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:35 smithi050 ceph-mon[33492]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:32:35.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/1267346570' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:32:37.190 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:37 smithi050 ceph-mon[33492]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-05T07:32:37.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:37 smithi043 ceph-mon[29879]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-05T07:32:37.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:37 smithi043 ceph-mon[41444]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-05T07:32:39.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:39 smithi050 ceph-mon[33492]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:32:39.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:39 smithi043 ceph-mon[29879]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:32:39.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:39 smithi043 ceph-mon[41444]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:32:41.114 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:41 smithi050 ceph-mon[33492]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:41.114 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:41 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-05T07:32:41.114 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:41 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:41.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:41 smithi043 ceph-mon[29879]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:41.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:41 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-05T07:32:41.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:41 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:41.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:41 smithi043 ceph-mon[41444]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:41.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:41 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-05T07:32:41.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:41 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:42.171 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:42 smithi050 ceph-mon[33492]: Deploying daemon osd.5 on smithi050 2024-02-05T07:32:42.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:42 smithi043 ceph-mon[29879]: Deploying daemon osd.5 on smithi050 2024-02-05T07:32:42.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:42 smithi043 ceph-mon[41444]: Deploying daemon osd.5 on smithi050 2024-02-05T07:32:43.132 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:43 smithi050 ceph-mon[33492]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:43.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:43 smithi043 ceph-mon[29879]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:43.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:43 smithi043 ceph-mon[41444]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:45.133 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:45 smithi050 ceph-mon[33492]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:45.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:45 smithi043 ceph-mon[29879]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:45.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:45 smithi043 ceph-mon[41444]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:47.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:46 smithi050 ceph-mon[33492]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:47.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:46 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:47.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:46 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:47.034 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:46 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[41444]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[29879]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:46 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[29879]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[41444]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:48 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:48 smithi050 ceph-mon[33492]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:48 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.528 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.527+0000 7f88627fc700 1 -- 172.21.15.50:0/3143788431 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8864062160 con 0x7f884c06d9d0 2024-02-05T07:32:49.528 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 5 on host 'smithi050' 2024-02-05T07:32:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f884c06d9d0 msgr2=0x7f884c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f884c06d9d0 0x7f884c06fe90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8854009550 tx=0x7f88540091b0 comp rx=0 tx=0).stop 2024-02-05T07:32:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 msgr2=0x7f8864100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864100cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f885c0036f0 tx=0x7f885c00f690 comp rx=0 tx=0).stop 2024-02-05T07:32:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 shutdown_connections 2024-02-05T07:32:49.530 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f884c06d9d0 0x7f884c06fe90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:49.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8864107a40 0x7f8864100cd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:49.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f88641027e0 0x7f88641020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:49.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 --2- 172.21.15.50:0/3143788431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8864105110 0x7f8864100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:49.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 >> 172.21.15.50:0/3143788431 conn(0x7f88640fc1f0 msgr2=0x7f8864103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:32:49.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 shutdown_connections 2024-02-05T07:32:49.531 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:49.529+0000 7f886b2a0700 1 -- 172.21.15.50:0/3143788431 wait complete. 2024-02-05T07:32:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:49 smithi050 ceph-mon[33492]: from='osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:32:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:49 smithi050 ceph-mon[33492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:32:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:49 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:49 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:49 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[29879]: from='osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:32:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[29879]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:32:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[41444]: from='osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:32:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[41444]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-05T07:32:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:49 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:32:50.182 DEBUG:teuthology.orchestra.run.smithi050:osd.5> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.5.service 2024-02-05T07:32:50.184 INFO:tasks.cephadm:Deploying osd.6 on smithi050 with /dev/vg_nvme/lv_2... 2024-02-05T07:32:50.184 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-05T07:32:50.948 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:32:51.000 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:50 smithi050 ceph-mon[33492]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:51.000 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:50 smithi050 ceph-mon[33492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-05T07:32:51.000 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:50 smithi050 ceph-mon[33492]: from='osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:51.000 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:50 smithi050 ceph-mon[33492]: osdmap e38: 6 total, 5 up, 6 in 2024-02-05T07:32:51.000 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:50 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:51.000 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:50 smithi050 ceph-mon[33492]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[29879]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[29879]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-05T07:32:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[29879]: from='osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[29879]: osdmap e38: 6 total, 5 up, 6 in 2024-02-05T07:32:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[29879]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:51.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[41444]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:51.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[41444]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-05T07:32:51.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[41444]: from='osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:51.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[41444]: osdmap e38: 6 total, 5 up, 6 in 2024-02-05T07:32:51.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:51.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:50 smithi043 ceph-mon[41444]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:32:51.253 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 07:32:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T07:32:50.999+0000 7f6b1693f700 -1 osd.5 0 waiting for initial osdmap 2024-02-05T07:32:51.253 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 07:32:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T07:32:51.003+0000 7f6b0e731700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:32:51.967 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:51 smithi050 ceph-mon[33492]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:32:51.967 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:51 smithi050 ceph-mon[33492]: osdmap e39: 6 total, 5 up, 6 in 2024-02-05T07:32:51.967 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:51.967 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:51 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:51.967 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:51 smithi050 ceph-mon[33492]: from='osd.5 ' entity='osd.5' 2024-02-05T07:32:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[29879]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:32:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[29879]: osdmap e39: 6 total, 5 up, 6 in 2024-02-05T07:32:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[29879]: from='osd.5 ' entity='osd.5' 2024-02-05T07:32:52.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[41444]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:32:52.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[41444]: osdmap e39: 6 total, 5 up, 6 in 2024-02-05T07:32:52.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:52.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:52.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:51 smithi043 ceph-mon[41444]: from='osd.5 ' entity='osd.5' 2024-02-05T07:32:52.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:52 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:32:52.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:52 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:32:52.906 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:52 smithi050 ceph-mon[33492]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:52.907 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:52 smithi050 ceph-mon[33492]: osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] boot 2024-02-05T07:32:52.907 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:52 smithi050 ceph-mon[33492]: osdmap e40: 6 total, 6 up, 6 in 2024-02-05T07:32:52.907 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:52 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:52.907 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:52 smithi050 ceph-mon[33492]: osdmap e41: 6 total, 6 up, 6 in 2024-02-05T07:32:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:32:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:32:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[29879]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[29879]: osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] boot 2024-02-05T07:32:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[29879]: osdmap e40: 6 total, 6 up, 6 in 2024-02-05T07:32:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[29879]: osdmap e41: 6 total, 6 up, 6 in 2024-02-05T07:32:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:32:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:32:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[41444]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-02-05T07:32:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[41444]: osd.5 [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] boot 2024-02-05T07:32:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[41444]: osdmap e40: 6 total, 6 up, 6 in 2024-02-05T07:32:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:32:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:52 smithi043 ceph-mon[41444]: osdmap e41: 6 total, 6 up, 6 in 2024-02-05T07:32:53.555 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-05T07:32:53.556 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-05T07:32:53.556 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-05T07:32:53.556 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-05T07:32:53.556 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00987094 s, 1.1 GB/s 2024-02-05T07:32:53.556 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-05T07:32:54.739 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_2 2024-02-05T07:32:54.980 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:32:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: osdmap e42: 6 total, 6 up, 6 in 2024-02-05T07:32:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: osdmap e42: 6 total, 6 up, 6 in 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:55.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:55.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:55.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:54 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 330 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: osdmap e42: 6 total, 6 up, 6 in 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:32:55.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:54 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:32:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:55 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi050 2024-02-05T07:32:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:55 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi050 to 8546M 2024-02-05T07:32:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:55 smithi043 ceph-mon[29879]: osdmap e43: 6 total, 6 up, 6 in 2024-02-05T07:32:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:55 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi050 2024-02-05T07:32:56.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:55 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi050 to 8546M 2024-02-05T07:32:56.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:55 smithi043 ceph-mon[41444]: osdmap e43: 6 total, 6 up, 6 in 2024-02-05T07:32:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:55 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi050 2024-02-05T07:32:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:55 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi050 to 8546M 2024-02-05T07:32:56.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:55 smithi050 ceph-mon[33492]: osdmap e43: 6 total, 6 up, 6 in 2024-02-05T07:32:56.593 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 -- 172.21.15.50:0/459461467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54102f50 msgr2=0x7ffa541033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/459461467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54102f50 0x7ffa541033b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffa480099c0 tx=0x7ffa48009cd0 comp rx=0 tx=0).stop 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 -- 172.21.15.50:0/459461467 shutdown_connections 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/459461467 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa541038f0 0x7ffa5410a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/459461467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa54101d50 0x7ffa54102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/459461467 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54102f50 0x7ffa541033b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 -- 172.21.15.50:0/459461467 >> 172.21.15.50:0/459461467 conn(0x7ffa540fd330 msgr2=0x7ffa540ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 -- 172.21.15.50:0/459461467 shutdown_connections 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 -- 172.21.15.50:0/459461467 wait complete. 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.592+0000 7ffa5a15c700 1 Processor -- start 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa5a15c700 1 -- start start 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa5a15c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54101d50 0x7ffa54075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:56.594 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa5a15c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa54102f50 0x7ffa54074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa5a15c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa541038f0 0x7ffa54074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa5a15c700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54074dd0 con 0x7ffa54102f50 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa5a15c700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa54074f40 con 0x7ffa54101d50 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa5a15c700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541a12c0 con 0x7ffa541038f0 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54101d50 0x7ffa54075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54101d50 0x7ffa54075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:56574/0 (socket says 172.21.15.50:56574) 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 -- 172.21.15.50:0/1592492050 learned_addr learned my addr 172.21.15.50:0/1592492050 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa52ffd700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa54102f50 0x7ffa54074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:56.595 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa53fff700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa541038f0 0x7ffa54074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 -- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa541038f0 msgr2=0x7ffa54074650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa541038f0 0x7ffa54074650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 -- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa54102f50 msgr2=0x7ffa54074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa54102f50 0x7ffa54074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 -- 172.21.15.50:0/1592492050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480095f0 con 0x7ffa54101d50 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.593+0000 7ffa537fe700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54101d50 0x7ffa54075ac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400d980 tx=0x7ffa4400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.594+0000 7ffa50ff9700 1 -- 172.21.15.50:0/1592492050 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4400f890 con 0x7ffa54101d50 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.594+0000 7ffa50ff9700 1 -- 172.21.15.50:0/1592492050 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffa4400fe70 con 0x7ffa54101d50 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.594+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa541a15a0 con 0x7ffa54101d50 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.594+0000 7ffa50ff9700 1 -- 172.21.15.50:0/1592492050 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44010e80 con 0x7ffa54101d50 2024-02-05T07:32:56.597 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.594+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa541a1af0 con 0x7ffa54101d50 2024-02-05T07:32:56.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.595+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa54108930 con 0x7ffa54101d50 2024-02-05T07:32:56.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.595+0000 7ffa50ff9700 1 -- 172.21.15.50:0/1592492050 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7ffa4400f9f0 con 0x7ffa54101d50 2024-02-05T07:32:56.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.595+0000 7ffa50ff9700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ffa3c06d9d0 0x7ffa3c06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:32:56.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.596+0000 7ffa50ff9700 1 -- 172.21.15.50:0/1592492050 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4423+0+0 (secure 0 0 0) 0x7ffa4408c510 con 0x7ffa54101d50 2024-02-05T07:32:56.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.596+0000 7ffa52ffd700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ffa3c06d9d0 0x7ffa3c06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:32:56.598 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.596+0000 7ffa52ffd700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ffa3c06d9d0 0x7ffa3c06fe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009990 tx=0x7ffa48000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:32:56.601 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.600+0000 7ffa50ff9700 1 -- 172.21.15.50:0/1592492050 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa440565b0 con 0x7ffa54101d50 2024-02-05T07:32:56.810 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:32:56.809+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa54062160 con 0x7ffa3c06d9d0 2024-02-05T07:32:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:56 smithi043 ceph-mon[29879]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:56 smithi043 ceph-mon[29879]: osdmap e44: 6 total, 6 up, 6 in 2024-02-05T07:32:57.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:56 smithi043 ceph-mon[41444]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:57.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:56 smithi043 ceph-mon[41444]: osdmap e44: 6 total, 6 up, 6 in 2024-02-05T07:32:57.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:56 smithi050 ceph-mon[33492]: pgmap v143: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 330 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:57.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:56 smithi050 ceph-mon[33492]: osdmap e44: 6 total, 6 up, 6 in 2024-02-05T07:32:58.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:57 smithi050 ceph-mon[33492]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:58.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:57 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:58.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:57 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:58.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:57 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[29879]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[41444]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:32:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:32:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:32:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:57 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:32:58.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:58 smithi050 ceph-mon[33492]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:58 smithi043 ceph-mon[29879]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:32:59.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:58 smithi043 ceph-mon[41444]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:00.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/1514283680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]: dispatch 2024-02-05T07:33:00.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]: dispatch 2024-02-05T07:33:00.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]': finished 2024-02-05T07:33:00.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[29879]: osdmap e45: 7 total, 6 up, 7 in 2024-02-05T07:33:00.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:00.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/1514283680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]: dispatch 2024-02-05T07:33:00.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]: dispatch 2024-02-05T07:33:00.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]': finished 2024-02-05T07:33:00.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[41444]: osdmap e45: 7 total, 6 up, 7 in 2024-02-05T07:33:00.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:32:59 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:00.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/1514283680' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]: dispatch 2024-02-05T07:33:00.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]: dispatch 2024-02-05T07:33:00.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b63e0e96-54e3-482b-91f7-4662d8856477"}]': finished 2024-02-05T07:33:00.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:59 smithi050 ceph-mon[33492]: osdmap e45: 7 total, 6 up, 7 in 2024-02-05T07:33:00.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:32:59 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:00 smithi043 ceph-mon[29879]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:01.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/1434590005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:33:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:00 smithi043 ceph-mon[41444]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/1434590005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:33:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:00 smithi050 ceph-mon[33492]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/1434590005' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:33:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:02 smithi043 ceph-mon[29879]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:03.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:02 smithi043 ceph-mon[41444]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:03.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:02 smithi050 ceph-mon[33492]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:05.301 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:04 smithi050 ceph-mon[33492]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:04 smithi043 ceph-mon[29879]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:05.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:04 smithi043 ceph-mon[41444]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:07.288 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:06 smithi050 ceph-mon[33492]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:07.288 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-05T07:33:07.288 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:07.288 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:06 smithi050 ceph-mon[33492]: Deploying daemon osd.6 on smithi050 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[29879]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[29879]: Deploying daemon osd.6 on smithi050 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[41444]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:07.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:06 smithi043 ceph-mon[41444]: Deploying daemon osd.6 on smithi050 2024-02-05T07:33:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:08 smithi043 ceph-mon[29879]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:09.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:08 smithi043 ceph-mon[41444]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:08 smithi050 ceph-mon[33492]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:10 smithi050 ceph-mon[33492]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:10 smithi043 ceph-mon[29879]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:11.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:10 smithi043 ceph-mon[41444]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:12.460 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:12 smithi050 ceph-mon[33492]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:12.460 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:12.460 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:12.461 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:12 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[29879]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[41444]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:12 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:14.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:14 smithi050 ceph-mon[33492]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:14.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:14.764 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:14 smithi043 ceph-mon[29879]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:14 smithi043 ceph-mon[41444]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:15.125 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.123+0000 7ffa50ff9700 1 -- 172.21.15.50:0/1592492050 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffa54062160 con 0x7ffa3c06d9d0 2024-02-05T07:33:15.126 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 6 on host 'smithi050' 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ffa3c06d9d0 msgr2=0x7ffa3c06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ffa3c06d9d0 0x7ffa3c06fe90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009990 tx=0x7ffa48000bc0 comp rx=0 tx=0).stop 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54101d50 msgr2=0x7ffa54075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54101d50 0x7ffa54075ac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa4400d980 tx=0x7ffa4400dc90 comp rx=0 tx=0).stop 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 shutdown_connections 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ffa3c06d9d0 0x7ffa3c06fe90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa541038f0 0x7ffa54074650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa54101d50 0x7ffa54075ac0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 --2- 172.21.15.50:0/1592492050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa54102f50 0x7ffa54074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 >> 172.21.15.50:0/1592492050 conn(0x7ffa540fd330 msgr2=0x7ffa540fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 shutdown_connections 2024-02-05T07:33:15.128 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:15.126+0000 7ffa5a15c700 1 -- 172.21.15.50:0/1592492050 wait complete. 2024-02-05T07:33:15.735 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:15 smithi050 ceph-mon[33492]: from='osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:33:15.735 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:15 smithi050 ceph-mon[33492]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:33:15.735 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:15 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:15.735 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:15 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:15.767 DEBUG:teuthology.orchestra.run.smithi050:osd.6> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.6.service 2024-02-05T07:33:15.769 INFO:tasks.cephadm:Deploying osd.7 on smithi050 with /dev/vg_nvme/lv_1... 2024-02-05T07:33:15.769 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[29879]: from='osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[29879]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[41444]: from='osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[41444]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:15 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:16.682 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-mon[33492]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:16.682 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-mon[33492]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-05T07:33:16.682 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-mon[33492]: from='osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:16.682 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-mon[33492]: osdmap e46: 7 total, 6 up, 7 in 2024-02-05T07:33:16.682 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:16.682 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-mon[33492]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[41444]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[41444]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-05T07:33:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[41444]: from='osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[41444]: osdmap e46: 7 total, 6 up, 7 in 2024-02-05T07:33:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[41444]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:16.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[29879]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:16.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[29879]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-05T07:33:16.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[29879]: from='osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:16.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[29879]: osdmap e46: 7 total, 6 up, 7 in 2024-02-05T07:33:16.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:16.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:16 smithi043 ceph-mon[29879]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:16.822 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:33:16.946 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T07:33:16.679+0000 7fb678528700 -1 osd.6 0 waiting for initial osdmap 2024-02-05T07:33:16.946 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 07:33:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T07:33:16.685+0000 7fb67031a700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-05T07:33:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:17 smithi050 ceph-mon[33492]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:33:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:17 smithi050 ceph-mon[33492]: osdmap e47: 7 total, 6 up, 7 in 2024-02-05T07:33:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:17 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:17 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:17 smithi050 ceph-mon[33492]: from='osd.6 ' entity='osd.6' 2024-02-05T07:33:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[29879]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:33:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[29879]: osdmap e47: 7 total, 6 up, 7 in 2024-02-05T07:33:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[29879]: from='osd.6 ' entity='osd.6' 2024-02-05T07:33:17.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[41444]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:33:17.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[41444]: osdmap e47: 7 total, 6 up, 7 in 2024-02-05T07:33:17.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:17.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:17.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:17 smithi043 ceph-mon[41444]: from='osd.6 ' entity='osd.6' 2024-02-05T07:33:18.674 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:18 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:33:18.674 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:18 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:33:18.674 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:18 smithi050 ceph-mon[33492]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:18.674 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:18 smithi050 ceph-mon[33492]: osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] boot 2024-02-05T07:33:18.674 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:18 smithi050 ceph-mon[33492]: osdmap e48: 7 total, 7 up, 7 in 2024-02-05T07:33:18.674 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:18 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:33:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:33:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[41444]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[41444]: osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] boot 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[41444]: osdmap e48: 7 total, 7 up, 7 in 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[29879]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 179 MiB used, 536 GiB / 536 GiB avail 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[29879]: osd.6 [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] boot 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[29879]: osdmap e48: 7 total, 7 up, 7 in 2024-02-05T07:33:18.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:33:19.543 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: osdmap e49: 7 total, 7 up, 7 in 2024-02-05T07:33:19.543 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.543 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.543 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.543 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:19.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:19.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.544 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:19 smithi050 ceph-mon[33492]: osdmap e50: 7 total, 7 up, 7 in 2024-02-05T07:33:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: osdmap e49: 7 total, 7 up, 7 in 2024-02-05T07:33:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[29879]: osdmap e50: 7 total, 7 up, 7 in 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: osdmap e49: 7 total, 7 up, 7 in 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:19.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:19.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:19.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:19.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:19 smithi043 ceph-mon[41444]: osdmap e50: 7 total, 7 up, 7 in 2024-02-05T07:33:19.991 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-05T07:33:19.992 INFO:teuthology.orchestra.run.smithi050.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-05T07:33:19.992 INFO:teuthology.orchestra.run.smithi050.stderr: stderr: 10+0 records in 2024-02-05T07:33:19.992 INFO:teuthology.orchestra.run.smithi050.stderr:10+0 records out 2024-02-05T07:33:19.992 INFO:teuthology.orchestra.run.smithi050.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00874721 s, 1.2 GB/s 2024-02-05T07:33:19.992 INFO:teuthology.orchestra.run.smithi050.stderr:--> Zapping successful for: 2024-02-05T07:33:20.548 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi050:vg_nvme/lv_1 2024-02-05T07:33:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 517 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:33:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi050 2024-02-05T07:33:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi050 to 5697M 2024-02-05T07:33:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:20 smithi050 ceph-mon[33492]: osdmap e51: 7 total, 7 up, 7 in 2024-02-05T07:33:20.791 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 517 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi050 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi050 to 5697M 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[29879]: osdmap e51: 7 total, 7 up, 7 in 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: pgmap v161: 1 pgs: 1 remapped+peering; 577 KiB data, 517 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi050 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi050 to 5697M 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:20 smithi043 ceph-mon[41444]: osdmap e51: 7 total, 7 up, 7 in 2024-02-05T07:33:22.186 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:21 smithi050 ceph-mon[33492]: osdmap e52: 7 total, 7 up, 7 in 2024-02-05T07:33:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:21 smithi043 ceph-mon[41444]: osdmap e52: 7 total, 7 up, 7 in 2024-02-05T07:33:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:21 smithi043 ceph-mon[29879]: osdmap e52: 7 total, 7 up, 7 in 2024-02-05T07:33:22.439 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 -- 172.21.15.50:0/1181462449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 msgr2=0x7f11881010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 --2- 172.21.15.50:0/1181462449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f11881010e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f117c0099c0 tx=0x7f117c009cd0 comp rx=0 tx=0).stop 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 -- 172.21.15.50:0/1181462449 shutdown_connections 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 --2- 172.21.15.50:0/1181462449 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1188102860 0x7f1188109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 --2- 172.21.15.50:0/1181462449 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f11881010e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 --2- 172.21.15.50:0/1181462449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1188101ec0 0x7f1188102320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 -- 172.21.15.50:0/1181462449 >> 172.21.15.50:0/1181462449 conn(0x7f11880fc280 msgr2=0x7f11880fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 -- 172.21.15.50:0/1181462449 shutdown_connections 2024-02-05T07:33:22.440 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 -- 172.21.15.50:0/1181462449 wait complete. 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 Processor -- start 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 -- start start 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f1188104c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.438+0000 7f118ecb2700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1188101ec0 0x7f11881032a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ecb2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1188102860 0x7f11881037e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ecb2700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1188103f60 con 0x7f1188101ec0 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ecb2700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11881040d0 con 0x7f1188100cc0 2024-02-05T07:33:22.442 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ecb2700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11881a11d0 con 0x7f1188102860 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f1188104c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f1188104c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:36952/0 (socket says 172.21.15.50:36952) 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 -- 172.21.15.50:0/2466901998 learned_addr learned my addr 172.21.15.50:0/2466901998 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f1187fff700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1188101ec0 0x7f11881032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118d24f700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1188102860 0x7f11881037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 -- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1188102860 msgr2=0x7f11881037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1188102860 0x7f11881037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 -- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1188101ec0 msgr2=0x7f11881032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1188101ec0 0x7f11881032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 -- 172.21.15.50:0/2466901998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0095f0 con 0x7f1188100cc0 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f118ca4e700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f1188104c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f117c009f50 tx=0x7f117c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.439+0000 7f1185ffb700 1 -- 172.21.15.50:0/2466901998 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c01d070 con 0x7f1188100cc0 2024-02-05T07:33:22.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.440+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11881a1450 con 0x7f1188100cc0 2024-02-05T07:33:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.440+0000 7f1185ffb700 1 -- 172.21.15.50:0/2466901998 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f117c022470 con 0x7f1188100cc0 2024-02-05T07:33:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.440+0000 7f1185ffb700 1 -- 172.21.15.50:0/2466901998 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117c0037c0 con 0x7f1188100cc0 2024-02-05T07:33:22.444 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.440+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11881a1940 con 0x7f1188100cc0 2024-02-05T07:33:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.441+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11880689e0 con 0x7f1188100cc0 2024-02-05T07:33:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.442+0000 7f1185ffb700 1 -- 172.21.15.50:0/2466901998 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f117c022890 con 0x7f1188100cc0 2024-02-05T07:33:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.442+0000 7f1185ffb700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f117806d9d0 0x7f117806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.442+0000 7f1185ffb700 1 -- 172.21.15.50:0/2466901998 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 4855+0+0 (secure 0 0 0) 0x7f117c08e170 con 0x7f1188100cc0 2024-02-05T07:33:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.442+0000 7f1187fff700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f117806d9d0 0x7f117806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:22.445 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.443+0000 7f1187fff700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f117806d9d0 0x7f117806fe90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f117400ac50 tx=0x7f117400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:22.448 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.447+0000 7f1185ffb700 1 -- 172.21.15.50:0/2466901998 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f117c058230 con 0x7f1188100cc0 2024-02-05T07:33:22.662 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:22.661+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f11880fde90 con 0x7f117806d9d0 2024-02-05T07:33:23.287 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:22 smithi050 ceph-mon[33492]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 517 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:33:23.287 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:33:23.287 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:33:23.287 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:22 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[29879]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 517 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[41444]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 517 MiB used, 625 GiB / 626 GiB avail 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-05T07:33:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:22 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:24.271 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:23 smithi050 ceph-mon[33492]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:33:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:23 smithi043 ceph-mon[29879]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:33:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:23 smithi043 ceph-mon[41444]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:33:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:24 smithi043 ceph-mon[29879]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-05T07:33:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:24 smithi043 ceph-mon[41444]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-05T07:33:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:24 smithi050 ceph-mon[33492]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-02-05T07:33:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/1331907644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]: dispatch 2024-02-05T07:33:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]: dispatch 2024-02-05T07:33:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]': finished 2024-02-05T07:33:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[29879]: osdmap e53: 8 total, 7 up, 8 in 2024-02-05T07:33:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/1331907644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]: dispatch 2024-02-05T07:33:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]: dispatch 2024-02-05T07:33:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]': finished 2024-02-05T07:33:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[41444]: osdmap e53: 8 total, 7 up, 8 in 2024-02-05T07:33:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:25 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/1331907644' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]: dispatch 2024-02-05T07:33:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]: dispatch 2024-02-05T07:33:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "91f453b0-c206-41a4-ae3c-422d2f3b8b0b"}]': finished 2024-02-05T07:33:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:25 smithi050 ceph-mon[33492]: osdmap e53: 8 total, 7 up, 8 in 2024-02-05T07:33:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:25 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:26 smithi043 ceph-mon[29879]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:33:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/819183429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:33:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:26 smithi043 ceph-mon[41444]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:33:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/819183429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:33:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:26 smithi050 ceph-mon[33492]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-05T07:33:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/819183429' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-05T07:33:29.032 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:28 smithi050 ceph-mon[33492]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-05T07:33:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:28 smithi043 ceph-mon[29879]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-05T07:33:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:28 smithi043 ceph-mon[41444]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-05T07:33:31.017 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:30 smithi050 ceph-mon[33492]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-05T07:33:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:30 smithi043 ceph-mon[29879]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-05T07:33:31.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:30 smithi043 ceph-mon[41444]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-02-05T07:33:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[29879]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:33:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-05T07:33:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[29879]: Deploying daemon osd.7 on smithi050 2024-02-05T07:33:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[41444]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:33:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-05T07:33:33.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:33.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:32 smithi043 ceph-mon[41444]: Deploying daemon osd.7 on smithi050 2024-02-05T07:33:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:32 smithi050 ceph-mon[33492]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-02-05T07:33:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-05T07:33:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:32 smithi050 ceph-mon[33492]: Deploying daemon osd.7 on smithi050 2024-02-05T07:33:35.249 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:34 smithi050 ceph-mon[33492]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:35.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:34 smithi043 ceph-mon[29879]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:35.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:34 smithi043 ceph-mon[41444]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:37.001 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:36 smithi050 ceph-mon[33492]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:37.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:36 smithi043 ceph-mon[29879]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:37.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:36 smithi043 ceph-mon[41444]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:38.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:38 smithi050 ceph-mon[33492]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:38.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:38 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:38.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:38 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:38.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:38 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:39.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[29879]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:39.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:39.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:39.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:39.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[41444]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:39.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:39.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:39.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:38 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:41.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:41.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: from='osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:33:41.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:33:41.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: from='osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:40 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:41.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: from='osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:33:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-05T07:33:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:40 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:41.479 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.477+0000 7f1185ffb700 1 -- 172.21.15.50:0/2466901998 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f11880fde90 con 0x7f117806d9d0 2024-02-05T07:33:41.479 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 7 on host 'smithi050' 2024-02-05T07:33:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f117806d9d0 msgr2=0x7f117806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f117806d9d0 0x7f117806fe90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f117400ac50 tx=0x7f117400a380 comp rx=0 tx=0).stop 2024-02-05T07:33:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 msgr2=0x7f1188104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f1188104c50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f117c009f50 tx=0x7f117c00f690 comp rx=0 tx=0).stop 2024-02-05T07:33:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 shutdown_connections 2024-02-05T07:33:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f117806d9d0 0x7f117806fe90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:41.481 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1188102860 0x7f11881037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1188100cc0 0x7f1188104c50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 --2- 172.21.15.50:0/2466901998 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1188101ec0 0x7f11881032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 >> 172.21.15.50:0/2466901998 conn(0x7f11880fc280 msgr2=0x7f11880fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 shutdown_connections 2024-02-05T07:33:41.482 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:41.480+0000 7f118ecb2700 1 -- 172.21.15.50:0/2466901998 wait complete. 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: from='osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: osdmap e54: 8 total, 7 up, 8 in 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:41 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:42.125 DEBUG:teuthology.orchestra.run.smithi050:osd.7> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.7.service 2024-02-05T07:33:42.168 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-05T07:33:42.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-05T07:33:42.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-05T07:33:42.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: from='osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:42.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: osdmap e54: 8 total, 7 up, 8 in 2024-02-05T07:33:42.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:42.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:42.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:42.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: from='osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: osdmap e54: 8 total, 7 up, 8 in 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]: dispatch 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:42.199 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:41 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:42.422 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:33:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[29879]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[29879]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:33:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[29879]: osdmap e55: 8 total, 7 up, 8 in 2024-02-05T07:33:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[41444]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[41444]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:33:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[41444]: osdmap e55: 8 total, 7 up, 8 in 2024-02-05T07:33:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:42 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:43.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:42 smithi050 ceph-mon[33492]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:43.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:42 smithi050 ceph-mon[33492]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-02-05T07:33:43.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:42 smithi050 ceph-mon[33492]: osdmap e55: 8 total, 7 up, 8 in 2024-02-05T07:33:43.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:43.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:42 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:43.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[29879]: purged_snaps scrub starts 2024-02-05T07:33:43.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[29879]: purged_snaps scrub ok 2024-02-05T07:33:43.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[29879]: osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] boot 2024-02-05T07:33:43.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[29879]: osdmap e56: 8 total, 8 up, 8 in 2024-02-05T07:33:43.946 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:44.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 -- 172.21.15.43:0/139131641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 msgr2=0x7f551c100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:44.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 --2- 172.21.15.43:0/139131641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 0x7f551c100270 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5510009990 tx=0x7f5510009ca0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 -- 172.21.15.43:0/139131641 shutdown_connections 2024-02-05T07:33:44.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 --2- 172.21.15.43:0/139131641 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f551c1007b0 0x7f551c10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 --2- 172.21.15.43:0/139131641 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c103370 0x7f551c103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 --2- 172.21.15.43:0/139131641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 0x7f551c100270 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 -- 172.21.15.43:0/139131641 >> 172.21.15.43:0/139131641 conn(0x7f551c0fb970 msgr2=0x7f551c0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:44.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 -- 172.21.15.43:0/139131641 shutdown_connections 2024-02-05T07:33:44.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 -- 172.21.15.43:0/139131641 wait complete. 2024-02-05T07:33:44.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[41444]: purged_snaps scrub starts 2024-02-05T07:33:44.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[41444]: purged_snaps scrub ok 2024-02-05T07:33:44.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[41444]: osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] boot 2024-02-05T07:33:44.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[41444]: osdmap e56: 8 total, 8 up, 8 in 2024-02-05T07:33:44.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:43 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:44.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.207+0000 7f5521b17700 1 Processor -- start 2024-02-05T07:33:44.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f5521b17700 1 -- start start 2024-02-05T07:33:44.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f5521b17700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 0x7f551c102a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f5521b17700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c1007b0 0x7f551c1010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f5521b17700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f551c103370 0x7f551c101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f5521b17700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c101d80 con 0x7f551c0ffdf0 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f5521b17700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c101ef0 con 0x7f551c103370 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f5521b17700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c1a1300 con 0x7f551c1007b0 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c1007b0 0x7f551c1010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c1007b0 0x7f551c1010c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:34856/0 (socket says 172.21.15.43:34856) 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551affd700 1 -- 172.21.15.43:0/1963053572 learned_addr learned my addr 172.21.15.43:0/1963053572 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551b7fe700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 0x7f551c102a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551bfff700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f551c103370 0x7f551c101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551b7fe700 1 -- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c1007b0 msgr2=0x7f551c1010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551b7fe700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c1007b0 0x7f551c1010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551b7fe700 1 -- 172.21.15.43:0/1963053572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f551c103370 msgr2=0x7f551c101600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.208+0000 7f551b7fe700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f551c103370 0x7f551c101600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f551b7fe700 1 -- 172.21.15.43:0/1963053572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c009520 con 0x7f551c0ffdf0 2024-02-05T07:33:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f551affd700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c1007b0 0x7f551c1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:33:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f551b7fe700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 0x7f551c102a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f550c00e9c0 tx=0x7f550c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f5518ff9700 1 -- 172.21.15.43:0/1963053572 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c00cb70 con 0x7f551c0ffdf0 2024-02-05T07:33:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f5518ff9700 1 -- 172.21.15.43:0/1963053572 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f550c00ee60 con 0x7f551c0ffdf0 2024-02-05T07:33:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55100095f0 con 0x7f551c0ffdf0 2024-02-05T07:33:44.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f5518ff9700 1 -- 172.21.15.43:0/1963053572 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550c018420 con 0x7f551c0ffdf0 2024-02-05T07:33:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.209+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c1a19a0 con 0x7f551c0ffdf0 2024-02-05T07:33:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.210+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c108af0 con 0x7f551c0ffdf0 2024-02-05T07:33:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.211+0000 7f5518ff9700 1 -- 172.21.15.43:0/1963053572 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f550c010430 con 0x7f551c0ffdf0 2024-02-05T07:33:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.211+0000 7f5518ff9700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f550406d9d0 0x7f550406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.211+0000 7f5518ff9700 1 -- 172.21.15.43:0/1963053572 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5316+0+0 (secure 0 0 0) 0x7f550c014070 con 0x7f551c0ffdf0 2024-02-05T07:33:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.211+0000 7f551affd700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f550406d9d0 0x7f550406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:44.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.211+0000 7f551affd700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f550406d9d0 0x7f550406fe90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55100058e0 tx=0x7f5510005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:44.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.216+0000 7f5518ff9700 1 -- 172.21.15.43:0/1963053572 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f550c056fc0 con 0x7f551c0ffdf0 2024-02-05T07:33:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:43 smithi050 ceph-mon[33492]: purged_snaps scrub starts 2024-02-05T07:33:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:43 smithi050 ceph-mon[33492]: purged_snaps scrub ok 2024-02-05T07:33:44.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:43 smithi050 ceph-mon[33492]: osd.7 [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] boot 2024-02-05T07:33:44.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:43 smithi050 ceph-mon[33492]: osdmap e56: 8 total, 8 up, 8 in 2024-02-05T07:33:44.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:43 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:33:44.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.416+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f551c10cd70 con 0x7f551c0ffdf0 2024-02-05T07:33:44.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.417+0000 7f5518ff9700 1 -- 172.21.15.43:0/1963053572 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v57) v1 ==== 74+0+130 (secure 0 0 0) 0x7f550c00ce60 con 0x7f551c0ffdf0 2024-02-05T07:33:44.418 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:33:44.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f550406d9d0 msgr2=0x7f550406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:44.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f550406d9d0 0x7f550406fe90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f55100058e0 tx=0x7f5510005960 comp rx=0 tx=0).stop 2024-02-05T07:33:44.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 msgr2=0x7f551c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:44.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 0x7f551c102a70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f550c00e9c0 tx=0x7f550c00ecd0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 shutdown_connections 2024-02-05T07:33:44.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f550406d9d0 0x7f550406fe90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f551c103370 0x7f551c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f551c0ffdf0 0x7f551c102a70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 --2- 172.21.15.43:0/1963053572 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f551c1007b0 0x7f551c1010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:44.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 >> 172.21.15.43:0/1963053572 conn(0x7f551c0fb970 msgr2=0x7f551c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:44.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 shutdown_connections 2024-02-05T07:33:44.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:44.419+0000 7f5521b17700 1 -- 172.21.15.43:0/1963053572 wait complete. 2024-02-05T07:33:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:44 smithi050 ceph-mon[33492]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:44 smithi050 ceph-mon[33492]: osdmap e57: 8 total, 8 up, 8 in 2024-02-05T07:33:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1963053572' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T07:33:45.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:44 smithi043 ceph-mon[29879]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:45.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:44 smithi043 ceph-mon[29879]: osdmap e57: 8 total, 8 up, 8 in 2024-02-05T07:33:45.116 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1963053572' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T07:33:45.117 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:44 smithi043 ceph-mon[41444]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-02-05T07:33:45.117 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:44 smithi043 ceph-mon[41444]: osdmap e57: 8 total, 8 up, 8 in 2024-02-05T07:33:45.117 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1963053572' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-05T07:33:45.144 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":57,"num_osds":8,"num_up_osds":8,"osd_up_since":1707118422,"num_in_osds":8,"osd_in_since":1707118405,"num_remapped_pgs":1} 2024-02-05T07:33:45.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T07:33:45.428 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:33:46.256 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: osdmap e58: 8 total, 8 up, 8 in 2024-02-05T07:33:46.256 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: osdmap e58: 8 total, 8 up, 8 in 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.257 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:46.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:45 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: osdmap e58: 8 total, 8 up, 8 in 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:45 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:47.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 -- 172.21.15.43:0/1903827443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed01018a0 msgr2=0x7f9ed01058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 --2- 172.21.15.43:0/1903827443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed01018a0 0x7f9ed01058f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80098f0 tx=0x7f9eb8009c00 comp rx=0 tx=0).stop 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 -- 172.21.15.43:0/1903827443 shutdown_connections 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 --2- 172.21.15.43:0/1903827443 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed0105fd0 0x7f9ed010dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 --2- 172.21.15.43:0/1903827443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed0100ef0 0x7f9ed01012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 --2- 172.21.15.43:0/1903827443 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed01018a0 0x7f9ed01058f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 -- 172.21.15.43:0/1903827443 >> 172.21.15.43:0/1903827443 conn(0x7f9ed0077d50 msgr2=0x7f9ed0078160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 -- 172.21.15.43:0/1903827443 shutdown_connections 2024-02-05T07:33:47.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.075+0000 7f9ed482e700 1 -- 172.21.15.43:0/1903827443 wait complete. 2024-02-05T07:33:47.077 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[29879]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:47.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[29879]: Detected new or changed devices on smithi050 2024-02-05T07:33:47.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi050 to 4273M 2024-02-05T07:33:47.078 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[29879]: osdmap e59: 8 total, 8 up, 8 in 2024-02-05T07:33:47.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[41444]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:47.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[41444]: Detected new or changed devices on smithi050 2024-02-05T07:33:47.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi050 to 4273M 2024-02-05T07:33:47.078 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:46 smithi043 ceph-mon[41444]: osdmap e59: 8 total, 8 up, 8 in 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 Processor -- start 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 -- start start 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed0100ef0 0x7f9ed010e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed01018a0 0x7f9ed010e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed0105fd0 0x7f9ed010ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01a5c70 con 0x7f9ed01018a0 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01a5de0 con 0x7f9ed0100ef0 2024-02-05T07:33:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ed482e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed01a5fa0 con 0x7f9ed0105fd0 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ecdd9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed01018a0 0x7f9ed010e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9eced9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed0105fd0 0x7f9ed010ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ecdd9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed01018a0 0x7f9ed010e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46486/0 (socket says 172.21.15.43:46486) 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ecdd9b700 1 -- 172.21.15.43:0/2062766737 learned_addr learned my addr 172.21.15.43:0/2062766737 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.076+0000 7f9ece59c700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed0100ef0 0x7f9ed010e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9eced9d700 1 -- 172.21.15.43:0/2062766737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed0100ef0 msgr2=0x7f9ed010e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9eced9d700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed0100ef0 0x7f9ed010e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9eced9d700 1 -- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed01018a0 msgr2=0x7f9ed010e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9eced9d700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed01018a0 0x7f9ed010e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9eced9d700 1 -- 172.21.15.43:0/2062766737 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec80095f0 con 0x7f9ed0105fd0 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9ecdd9b700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed01018a0 0x7f9ed010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9ece59c700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed0100ef0 0x7f9ed010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9eced9d700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed0105fd0 0x7f9ed010ed70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000efc0 tx=0x7f9ec000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9ec77fe700 1 -- 172.21.15.43:0/2062766737 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec0018070 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9ec77fe700 1 -- 172.21.15.43:0/2062766737 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f9ec000f040 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb8009520 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.077+0000 7f9ec77fe700 1 -- 172.21.15.43:0/2062766737 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec000cc50 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.078+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed01a65e0 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.079+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed0109290 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.079+0000 7f9ec77fe700 1 -- 172.21.15.43:0/2062766737 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f9ec0007500 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.079+0000 7f9ec77fe700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f9ebc06d9d0 0x7f9ebc06fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.079+0000 7f9ec77fe700 1 -- 172.21.15.43:0/2062766737 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f9ec008c660 con 0x7f9ed0105fd0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.079+0000 7f9ece59c700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f9ebc06d9d0 0x7f9ebc06fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.080+0000 7f9ece59c700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f9ebc06d9d0 0x7f9ebc06fe90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8005fd0 tx=0x7f9ec8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:47.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.084+0000 7f9ec77fe700 1 -- 172.21.15.43:0/2062766737 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9ec00563a0 con 0x7f9ed0105fd0 2024-02-05T07:33:47.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.288+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ed010f820 con 0x7f9ed0105fd0 2024-02-05T07:33:47.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.288+0000 7f9ec77fe700 1 -- 172.21.15.43:0/2062766737 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+13520 (secure 0 0 0) 0x7f9ec0059a60 con 0x7f9ed0105fd0 2024-02-05T07:33:47.290 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:33:47.290 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":60,"fsid":"f211dd04-c3f7-11ee-95b6-87774f69a715","created":"2024-02-05T07:28:14.907995+0000","modified":"2024-02-05T07:33:46.952773+0000","last_up_change":"2024-02-05T07:33:42.934794+0000","last_in_change":"2024-02-05T07:33:25.480400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T07:31:37.376556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c0b3234-6471-45a1-ad20-a2e96fd69264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6803","nonce":2715902880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6805","nonce":2715902880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6809","nonce":2715902880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6807","nonce":2715902880}]},"public_addr":"172.21.15.43:6803/2715902880","cluster_addr":"172.21.15.43:6805/2715902880","heartbeat_back_addr":"172.21.15.43:6809/2715902880","heartbeat_front_addr":"172.21.15.43:6807/2715902880","state":["exists","up"]},{"osd":1,"uuid":"903ff5b0-7d34-4899-8a55-50719f1a09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6811","nonce":193417229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6813","nonce":193417229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6817","nonce":193417229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6815","nonce":193417229}]},"public_addr":"172.21.15.43:6811/193417229","cluster_addr":"172.21.15.43:6813/193417229","heartbeat_back_addr":"172.21.15.43:6817/193417229","heartbeat_front_addr":"172.21.15.43:6815/193417229","state":["exists","up"]},{"osd":2,"uuid":"3527ab76-91de-487e-b5b1-6b5c1933b880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6819","nonce":2847356121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6821","nonce":2847356121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6825","nonce":2847356121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6823","nonce":2847356121}]},"public_addr":"172.21.15.43:6819/2847356121","cluster_addr":"172.21.15.43:6821/2847356121","heartbeat_back_addr":"172.21.15.43:6825/2847356121","heartbeat_front_addr":"172.21.15.43:6823/2847356121","state":["exists","up"]},{"osd":3,"uuid":"2a1431fa-3b8a-4aec-aaf4-58fa8ff28175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6827","nonce":4103809378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6829","nonce":4103809378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6833","nonce":4103809378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6831","nonce":4103809378}]},"public_addr":"172.21.15.43:6827/4103809378","cluster_addr":"172.21.15.43:6829/4103809378","heartbeat_back_addr":"172.21.15.43:6833/4103809378","heartbeat_front_addr":"172.21.15.43:6831/4103809378","state":["exists","up"]},{"osd":4,"uuid":"79c3dcf9-499c-4223-a1ce-b8cace5dc16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6801","nonce":2018737057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6803","nonce":2018737057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6807","nonce":2018737057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6805","nonce":2018737057}]},"public_addr":"172.21.15.50:6801/2018737057","cluster_addr":"172.21.15.50:6803/2018737057","heartbeat_back_addr":"172.21.15.50:6807/2018737057","heartbeat_front_addr":"172.21.15.50:6805/2018737057","state":["exists","up"]},{"osd":5,"uuid":"28fb0eeb-fce2-43e1-b5ee-e298db27ea98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6809","nonce":1047482547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6811","nonce":1047482547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6815","nonce":1047482547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6813","nonce":1047482547}]},"public_addr":"172.21.15.50:6809/1047482547","cluster_addr":"172.21.15.50:6811/1047482547","heartbeat_back_addr":"172.21.15.50:6815/1047482547","heartbeat_front_addr":"172.21.15.50:6813/1047482547","state":["exists","up"]},{"osd":6,"uuid":"b63e0e96-54e3-482b-91f7-4662d8856477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6817","nonce":2200245577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6819","nonce":2200245577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6823","nonce":2200245577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6821","nonce":2200245577}]},"public_addr":"172.21.15.50:6817/2200245577","cluster_addr":"172.21.15.50:6819/2200245577","heartbeat_back_addr":"172.21.15.50:6823/2200245577","heartbeat_front_addr":"172.21.15.50:6821/2200245577","state":["exists","up"]},{"osd":7,"uuid":"91f453b0-c206-41a4-ae3c-422d2f3b8b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6825","nonce":1826641620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6827","nonce":1826641620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6831","nonce":1826641620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6829","nonce":1826641620}]},"public_addr":"172.21.15.50:6825/1826641620","cluster_addr":"172.21.15.50:6827/1826641620","heartbeat_back_addr":"172.21.15.50:6831/1826641620","heartbeat_front_addr":"172.21.15.50:6829/1826641620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:30:41.016171+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:06.930315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:32.938609+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:59.536430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:32:23.858054+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:32:49.859628+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:33:15.590364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:33:41.632379+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3899220204":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6800/4155203878":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6801/4155203878":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6801/1476147457":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/1315802129":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6800/1476147457":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/1334642393":"2024-02-06T07:29:15.031939+0000","172.21.15.43:0/2189512325":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/3337674052":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/2734005924":"2024-02-06T07:28:47.869985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T07:33:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f9ebc06d9d0 msgr2=0x7f9ebc06fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f9ebc06d9d0 0x7f9ebc06fe90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8005fd0 tx=0x7f9ec8009310 comp rx=0 tx=0).stop 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed0105fd0 msgr2=0x7f9ed010ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed0105fd0 0x7f9ed010ed70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9ec000efc0 tx=0x7f9ec000c3a0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 shutdown_connections 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f9ebc06d9d0 0x7f9ebc06fe90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9ed0105fd0 0x7f9ed010ed70 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9ed0100ef0 0x7f9ed010e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 --2- 172.21.15.43:0/2062766737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9ed01018a0 0x7f9ed010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.290+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 >> 172.21.15.43:0/2062766737 conn(0x7f9ed0077d50 msgr2=0x7f9ed00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.291+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 shutdown_connections 2024-02-05T07:33:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:47.291+0000 7f9ed482e700 1 -- 172.21.15.43:0/2062766737 wait complete. 2024-02-05T07:33:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:46 smithi050 ceph-mon[33492]: pgmap v183: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:46 smithi050 ceph-mon[33492]: Detected new or changed devices on smithi050 2024-02-05T07:33:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:46 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi050 to 4273M 2024-02-05T07:33:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:46 smithi050 ceph-mon[33492]: osdmap e59: 8 total, 8 up, 8 in 2024-02-05T07:33:47.943 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-05T07:31:37.376556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-05T07:33:47.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-05T07:33:48.186 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:33:48.306 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:47 smithi043 ceph-mon[29879]: osdmap e60: 8 total, 8 up, 8 in 2024-02-05T07:33:48.307 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2062766737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:33:48.307 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:47 smithi043 ceph-mon[41444]: osdmap e60: 8 total, 8 up, 8 in 2024-02-05T07:33:48.307 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2062766737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:33:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:47 smithi050 ceph-mon[33492]: osdmap e60: 8 total, 8 up, 8 in 2024-02-05T07:33:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2062766737' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:33:49.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:48 smithi043 ceph-mon[29879]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:49.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:48 smithi043 ceph-mon[41444]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:49.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:48 smithi050 ceph-mon[33492]: pgmap v186: 1 pgs: 1 remapped+peering; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 -- 172.21.15.43:0/2223045573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68105f00 msgr2=0x7f0a6810dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 --2- 172.21.15.43:0/2223045573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68105f00 0x7f0a6810dad0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000b0a0 tx=0x7f0a6000b3b0 comp rx=0 tx=0).stop 2024-02-05T07:33:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 -- 172.21.15.43:0/2223045573 shutdown_connections 2024-02-05T07:33:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 --2- 172.21.15.43:0/2223045573 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68105f00 0x7f0a6810dad0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 --2- 172.21.15.43:0/2223045573 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a68100eb0 0x7f0a68101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 --2- 172.21.15.43:0/2223045573 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68101860 0x7f0a68105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 -- 172.21.15.43:0/2223045573 >> 172.21.15.43:0/2223045573 conn(0x7f0a68076c70 msgr2=0x7f0a68077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 -- 172.21.15.43:0/2223045573 shutdown_connections 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 -- 172.21.15.43:0/2223045573 wait complete. 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 Processor -- start 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.931+0000 7f0a6df84700 1 -- start start 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a6df84700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68100eb0 0x7f0a681028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a6df84700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a68101860 0x7f0a68102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a6df84700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68105f00 0x7f0a68103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a6df84700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681a5c70 con 0x7f0a68100eb0 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a6df84700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681a5de0 con 0x7f0a68101860 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a6df84700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681a5fa0 con 0x7f0a68105f00 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a677fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68100eb0 0x7f0a681028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68105f00 0x7f0a68103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:49.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68105f00 0x7f0a68103340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:34890/0 (socket says 172.21.15.43:34890) 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 -- 172.21.15.43:0/1852060236 learned_addr learned my addr 172.21.15.43:0/1852060236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a66ffd700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a68101860 0x7f0a68102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 -- 172.21.15.43:0/1852060236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a68101860 msgr2=0x7f0a68102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a68101860 0x7f0a68102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 -- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68100eb0 msgr2=0x7f0a681028c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68100eb0 0x7f0a681028c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 -- 172.21.15.43:0/1852060236 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5c009520 con 0x7f0a68105f00 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a66ffd700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a68101860 0x7f0a68102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a677fe700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68100eb0 0x7f0a681028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a67fff700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68105f00 0x7f0a68103340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000b070 tx=0x7f0a60007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a64ff9700 1 -- 172.21.15.43:0/1852060236 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6000e040 con 0x7f0a68105f00 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a64ff9700 1 -- 172.21.15.43:0/1852060236 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0a6000be40 con 0x7f0a68105f00 2024-02-05T07:33:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.932+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a60009d00 con 0x7f0a68105f00 2024-02-05T07:33:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.933+0000 7f0a64ff9700 1 -- 172.21.15.43:0/1852060236 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60005040 con 0x7f0a68105f00 2024-02-05T07:33:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.933+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a681a6500 con 0x7f0a68105f00 2024-02-05T07:33:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.933+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6810a230 con 0x7f0a68105f00 2024-02-05T07:33:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.934+0000 7f0a64ff9700 1 -- 172.21.15.43:0/1852060236 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f0a6001d470 con 0x7f0a68105f00 2024-02-05T07:33:49.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.934+0000 7f0a64ff9700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f0a5006d9d0 0x7f0a5006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.934+0000 7f0a64ff9700 1 -- 172.21.15.43:0/1852060236 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f0a6008e3a0 con 0x7f0a68105f00 2024-02-05T07:33:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.934+0000 7f0a677fe700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f0a5006d9d0 0x7f0a5006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:49.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.935+0000 7f0a677fe700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f0a5006d9d0 0x7f0a5006fe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a58005950 tx=0x7f0a5800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:49.939+0000 7f0a64ff9700 1 -- 172.21.15.43:0/1852060236 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a60058060 con 0x7f0a68105f00 2024-02-05T07:33:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.144+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a68103df0 con 0x7f0a68105f00 2024-02-05T07:33:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.145+0000 7f0a64ff9700 1 -- 172.21.15.43:0/1852060236 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v60) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a6005b720 con 0x7f0a68105f00 2024-02-05T07:33:50.146 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-02-05T07:33:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f0a5006d9d0 msgr2=0x7f0a5006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f0a5006d9d0 0x7f0a5006fe90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0a58005950 tx=0x7f0a5800b410 comp rx=0 tx=0).stop 2024-02-05T07:33:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68105f00 msgr2=0x7f0a68103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:50.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68105f00 0x7f0a68103340 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0a6000b070 tx=0x7f0a60007930 comp rx=0 tx=0).stop 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 shutdown_connections 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f0a5006d9d0 0x7f0a5006fe90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0a68105f00 0x7f0a68103340 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a68100eb0 0x7f0a681028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 --2- 172.21.15.43:0/1852060236 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0a68101860 0x7f0a68102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 >> 172.21.15.43:0/1852060236 conn(0x7f0a68076c70 msgr2=0x7f0a6806a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 shutdown_connections 2024-02-05T07:33:50.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:33:50.147+0000 7f0a6df84700 1 -- 172.21.15.43:0/1852060236 wait complete. 2024-02-05T07:33:50.896 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi043 2024-02-05T07:33:50.896 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi043=foo.a' 2024-02-05T07:33:51.152 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:33:51.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:50 smithi050 ceph-mon[33492]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-02-05T07:33:51.273 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1852060236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T07:33:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:50 smithi043 ceph-mon[29879]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-02-05T07:33:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1852060236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T07:33:51.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:50 smithi043 ceph-mon[41444]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-02-05T07:33:51.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1852060236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-05T07:33:52.683 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.679+0000 7f27e3c56700 1 -- 172.21.15.50:0/1274148604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc0ffdf0 msgr2=0x7f27dc100270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.679+0000 7f27e3c56700 1 --2- 172.21.15.50:0/1274148604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc0ffdf0 0x7f27dc100270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009990 tx=0x7f27cc009ca0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.679+0000 7f27e3c56700 1 -- 172.21.15.50:0/1274148604 shutdown_connections 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.679+0000 7f27e3c56700 1 --2- 172.21.15.50:0/1274148604 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27dc1007b0 0x7f27dc10a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.679+0000 7f27e3c56700 1 --2- 172.21.15.50:0/1274148604 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27dc103370 0x7f27dc103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.679+0000 7f27e3c56700 1 --2- 172.21.15.50:0/1274148604 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc0ffdf0 0x7f27dc100270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.679+0000 7f27e3c56700 1 -- 172.21.15.50:0/1274148604 >> 172.21.15.50:0/1274148604 conn(0x7f27dc0fb970 msgr2=0x7f27dc0fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 -- 172.21.15.50:0/1274148604 shutdown_connections 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 -- 172.21.15.50:0/1274148604 wait complete. 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 Processor -- start 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 -- start start 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27dc0ffdf0 0x7f27dc197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:52.684 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc1007b0 0x7f27dc197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27dc103370 0x7f27dc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc19ee20 con 0x7f27dc0ffdf0 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc19ef60 con 0x7f27dc1007b0 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e3c56700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27dc19f0f0 con 0x7f27dc103370 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e11f1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc1007b0 0x7f27dc197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e11f1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc1007b0 0x7f27dc197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49616/0 (socket says 172.21.15.50:49616) 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e11f1700 1 -- 172.21.15.50:0/4066729698 learned_addr learned my addr 172.21.15.50:0/4066729698 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.680+0000 7f27e19f2700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27dc0ffdf0 0x7f27dc197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e21f3700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27dc103370 0x7f27dc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e11f1700 1 -- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27dc103370 msgr2=0x7f27dc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e11f1700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27dc103370 0x7f27dc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e11f1700 1 -- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27dc0ffdf0 msgr2=0x7f27dc197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e11f1700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27dc0ffdf0 0x7f27dc197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e11f1700 1 -- 172.21.15.50:0/4066729698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0095f0 con 0x7f27dc1007b0 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e11f1700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc1007b0 0x7f27dc197840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098e0 tx=0x7f27cc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:52.685 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27d2ffd700 1 -- 172.21.15.50:0/4066729698 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc01d070 con 0x7f27dc1007b0 2024-02-05T07:33:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27d2ffd700 1 -- 172.21.15.50:0/4066729698 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f27cc00fe20 con 0x7f27dc1007b0 2024-02-05T07:33:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27dc19f370 con 0x7f27dc1007b0 2024-02-05T07:33:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27d2ffd700 1 -- 172.21.15.50:0/4066729698 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc022bc0 con 0x7f27dc1007b0 2024-02-05T07:33:52.686 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.681+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27dc19f7e0 con 0x7f27dc1007b0 2024-02-05T07:33:52.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.682+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27dc108af0 con 0x7f27dc1007b0 2024-02-05T07:33:52.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.683+0000 7f27d2ffd700 1 -- 172.21.15.50:0/4066729698 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f27cc00f9a0 con 0x7f27dc1007b0 2024-02-05T07:33:52.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.683+0000 7f27d2ffd700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f27c806d9d0 0x7f27c806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:52.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.683+0000 7f27d2ffd700 1 -- 172.21.15.50:0/4066729698 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f27cc08e370 con 0x7f27dc1007b0 2024-02-05T07:33:52.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.683+0000 7f27e19f2700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f27c806d9d0 0x7f27c806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:52.687 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.684+0000 7f27e19f2700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f27c806d9d0 0x7f27c806fe90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27dc06a1a0 tx=0x7f27d800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:52.689 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.688+0000 7f27d2ffd700 1 -- 172.21.15.50:0/4066729698 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f27cc0580b0 con 0x7f27dc1007b0 2024-02-05T07:33:52.903 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.901+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f27dc10cde0 con 0x7f27c806d9d0 2024-02-05T07:33:52.910 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.909+0000 7f27d2ffd700 1 -- 172.21.15.50:0/4066729698 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f27dc10cde0 con 0x7f27c806d9d0 2024-02-05T07:33:52.910 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled rgw.foo.a update... 2024-02-05T07:33:52.912 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f27c806d9d0 msgr2=0x7f27c806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f27c806d9d0 0x7f27c806fe90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f27dc06a1a0 tx=0x7f27d800b410 comp rx=0 tx=0).stop 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc1007b0 msgr2=0x7f27dc197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc1007b0 0x7f27dc197840 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0098e0 tx=0x7f27cc009580 comp rx=0 tx=0).stop 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 shutdown_connections 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f27c806d9d0 0x7f27c806fe90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f27dc103370 0x7f27dc19e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f27dc0ffdf0 0x7f27dc197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 --2- 172.21.15.50:0/4066729698 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f27dc1007b0 0x7f27dc197840 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 >> 172.21.15.50:0/4066729698 conn(0x7f27dc0fb970 msgr2=0x7f27dc0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 shutdown_connections 2024-02-05T07:33:52.913 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:52.911+0000 7f27e3c56700 1 -- 172.21.15.50:0/4066729698 wait complete. 2024-02-05T07:33:53.238 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:52 smithi050 ceph-mon[33492]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-05T07:33:53.238 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:52 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:53.238 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:52 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:52 smithi043 ceph-mon[41444]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-05T07:33:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:52 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:52 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:52 smithi043 ceph-mon[29879]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-05T07:33:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:52 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:52 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:53.551 DEBUG:teuthology.orchestra.run.smithi043:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@rgw.foo.a.service 2024-02-05T07:33:53.554 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi050 2024-02-05T07:33:53.554 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-02-05T07:33:53.797 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:54.183 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:53 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:54.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:33:54.278 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-02-05T07:33:54.279 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:54.279 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:54.279 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:54.279 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-05T07:33:54.279 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-05T07:33:54.279 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:54.279 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:53 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:55.234 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:54 smithi050 ceph-mon[33492]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:33:55.235 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:54 smithi050 ceph-mon[33492]: Deploying daemon rgw.foo.a on smithi043 2024-02-05T07:33:55.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:54 smithi043 ceph-mon[29879]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:33:55.265 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:54 smithi043 ceph-mon[29879]: Deploying daemon rgw.foo.a on smithi043 2024-02-05T07:33:55.265 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:54 smithi043 ceph-mon[41444]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-05T07:33:55.265 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:54 smithi043 ceph-mon[41444]: Deploying daemon rgw.foo.a on smithi043 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/172727252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4101ec0 msgr2=0x7ff0e4102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/172727252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4101ec0 0x7ff0e4102320 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff0d40099c0 tx=0x7ff0d4009cd0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/172727252 shutdown_connections 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/172727252 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0e4102860 0x7ff0e4109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/172727252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0e4100cc0 0x7ff0e41010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/172727252 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4101ec0 0x7ff0e4102320 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/172727252 >> 172.21.15.50:0/172727252 conn(0x7ff0e40fc280 msgr2=0x7ff0e40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/172727252 shutdown_connections 2024-02-05T07:33:55.483 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.481+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/172727252 wait complete. 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 Processor -- start 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 -- start start 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0e4100cc0 0x7ff0e4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0e4101ec0 0x7ff0e4197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4102860 0x7ff0e419e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e419ed90 con 0x7ff0e4100cc0 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e419eed0 con 0x7ff0e4102860 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ebdc8700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e419f060 con 0x7ff0e4101ec0 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4102860 0x7ff0e419e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0e9b64700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0e4100cc0 0x7ff0e4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0e9363700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0e4101ec0 0x7ff0e4197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:55.485 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4102860 0x7ff0e419e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49630/0 (socket says 172.21.15.50:49630) 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0e9b64700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0e4100cc0 0x7ff0e4197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.50:43844/0 (socket says 172.21.15.50:43844) 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 -- 172.21.15.50:0/260481376 learned_addr learned my addr 172.21.15.50:0/260481376 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 -- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0e4101ec0 msgr2=0x7ff0e4197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0e4101ec0 0x7ff0e4197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 -- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0e4100cc0 msgr2=0x7ff0e4197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0e4100cc0 0x7ff0e4197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.482+0000 7ff0ea365700 1 -- 172.21.15.50:0/260481376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d40095f0 con 0x7ff0e4102860 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.483+0000 7ff0ea365700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4102860 0x7ff0e419e7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00b970 tx=0x7ff0dc00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.483+0000 7ff0daffd700 1 -- 172.21.15.50:0/260481376 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00d860 con 0x7ff0e4102860 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.483+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e419f340 con 0x7ff0e4102860 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.483+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e419f840 con 0x7ff0e4102860 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.483+0000 7ff0daffd700 1 -- 172.21.15.50:0/260481376 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff0dc0098b0 con 0x7ff0e4102860 2024-02-05T07:33:55.486 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.483+0000 7ff0daffd700 1 -- 172.21.15.50:0/260481376 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0dc00ed60 con 0x7ff0e4102860 2024-02-05T07:33:55.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.484+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c8005150 con 0x7ff0e4102860 2024-02-05T07:33:55.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.484+0000 7ff0daffd700 1 -- 172.21.15.50:0/260481376 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7ff0dc0203f0 con 0x7ff0e4102860 2024-02-05T07:33:55.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.485+0000 7ff0daffd700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff0d006d9d0 0x7ff0d006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:33:55.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.485+0000 7ff0daffd700 1 -- 172.21.15.50:0/260481376 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5287+0+0 (secure 0 0 0) 0x7ff0dc006e40 con 0x7ff0e4102860 2024-02-05T07:33:55.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.485+0000 7ff0e9b64700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff0d006d9d0 0x7ff0d006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:33:55.487 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.485+0000 7ff0e9b64700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff0d006d9d0 0x7ff0d006fe90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4101d20 tx=0x7ff0e000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:33:55.491 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.489+0000 7ff0daffd700 1 -- 172.21.15.50:0/260481376 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff0dc0568a0 con 0x7ff0e4102860 2024-02-05T07:33:55.692 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.691+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7ff0c80056d0 con 0x7ff0e4102860 2024-02-05T07:33:55.974 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 07:33:55 smithi043 systemd[1]: Starting Ceph rgw.foo.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:33:55.980 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.979+0000 7ff0daffd700 1 -- 172.21.15.50:0/260481376 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v61) v1 ==== 160+0+0 (secure 0 0 0) 0x7ff0dc056070 con 0x7ff0e4102860 2024-02-05T07:33:55.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff0d006d9d0 msgr2=0x7ff0d006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:55.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff0d006d9d0 0x7ff0d006fe90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4101d20 tx=0x7ff0e000a3b0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4102860 msgr2=0x7ff0e419e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:33:55.983 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4102860 0x7ff0e419e7c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc00b970 tx=0x7ff0dc00bc80 comp rx=0 tx=0).stop 2024-02-05T07:33:55.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 shutdown_connections 2024-02-05T07:33:55.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7ff0d006d9d0 0x7ff0d006fe90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0e4102860 0x7ff0e419e7c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0e4100cc0 0x7ff0e4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 --2- 172.21.15.50:0/260481376 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0e4101ec0 0x7ff0e4197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:33:55.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.981+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 >> 172.21.15.50:0/260481376 conn(0x7ff0e40fc280 msgr2=0x7ff0e40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:33:55.984 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.982+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 shutdown_connections 2024-02-05T07:33:55.985 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:33:55.982+0000 7ff0ebdc8700 1 -- 172.21.15.50:0/260481376 wait complete. 2024-02-05T07:33:55.985 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'datapool' created 2024-02-05T07:33:56.285 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/260481376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:33:56.285 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:33:56.312 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/260481376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:33:56.312 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:33:56.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/260481376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:33:56.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-05T07:33:56.313 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 07:33:55 smithi043 podman[69752]: 2024-02-05 07:33:55.876109017 +0000 UTC m=+0.017783880 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T07:33:56.313 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 07:33:55 smithi043 podman[69752]: 2024-02-05 07:33:55.98323366 +0000 UTC m=+0.124908516 container create 1e8aad3b997601eff37eaa5153a2467d6fe539eabc1d333ed175afdad3bf278f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T07:33:56.313 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 07:33:56 smithi043 podman[69752]: 2024-02-05 07:33:56.203536735 +0000 UTC m=+0.345211592 container init 1e8aad3b997601eff37eaa5153a2467d6fe539eabc1d333ed175afdad3bf278f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-05T07:33:56.313 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 07:33:56 smithi043 podman[69752]: 2024-02-05 07:33:56.208891189 +0000 UTC m=+0.350566045 container start 1e8aad3b997601eff37eaa5153a2467d6fe539eabc1d333ed175afdad3bf278f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-05T07:33:56.563 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 07:33:56 smithi043 bash[69752]: 1e8aad3b997601eff37eaa5153a2467d6fe539eabc1d333ed175afdad3bf278f 2024-02-05T07:33:56.563 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 07:33:56 smithi043 systemd[1]: Started Ceph rgw.foo.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:33:56.615 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- rbd pool init datapool 2024-02-05T07:33:56.857 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:33:57.270 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-05T07:33:57.270 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-05T07:33:57.270 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: osdmap e61: 8 total, 8 up, 8 in 2024-02-05T07:33:57.270 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.270 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.271 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.271 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.271 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.271 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:56 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: osdmap e61: 8 total, 8 up, 8 in 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: osdmap e61: 8 total, 8 up, 8 in 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:57.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:56 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:33:58.269 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:57 smithi050 ceph-mon[33492]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-02-05T07:33:58.269 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:57 smithi050 ceph-mon[33492]: osdmap e62: 8 total, 8 up, 8 in 2024-02-05T07:33:58.269 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2620616334' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:33:58.269 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[29879]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[29879]: osdmap e62: 8 total, 8 up, 8 in 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2620616334' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[41444]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[41444]: osdmap e62: 8 total, 8 up, 8 in 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2620616334' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:33:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-05T07:33:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: pgmap v193: 36 pgs: 1 creating+peering, 1 active+clean, 34 unknown; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: osdmap e63: 8 total, 8 up, 8 in 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/1926906586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: pgmap v193: 36 pgs: 1 creating+peering, 1 active+clean, 34 unknown; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: osdmap e63: 8 total, 8 up, 8 in 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:59.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:59.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:59.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/1926906586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:33:59.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: pgmap v193: 36 pgs: 1 creating+peering, 1 active+clean, 34 unknown; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: osdmap e63: 8 total, 8 up, 8 in 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/1926906586' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:33:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[41444]: Checking dashboard <-> RGW credentials 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[41444]: osdmap e64: 8 total, 8 up, 8 in 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[29879]: Checking dashboard <-> RGW credentials 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-05T07:34:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[29879]: osdmap e64: 8 total, 8 up, 8 in 2024-02-05T07:34:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:33:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:59 smithi050 ceph-mon[33492]: Checking dashboard <-> RGW credentials 2024-02-05T07:34:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-02-05T07:34:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:59 smithi050 ceph-mon[33492]: osdmap e64: 8 total, 8 up, 8 in 2024-02-05T07:34:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:33:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-05T07:34:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[29879]: pgmap v196: 68 pgs: 14 active+clean, 9 creating+peering, 45 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-05T07:34:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:34:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:34:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[29879]: osdmap e65: 8 total, 8 up, 8 in 2024-02-05T07:34:01.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[41444]: pgmap v196: 68 pgs: 14 active+clean, 9 creating+peering, 45 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-05T07:34:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:34:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:34:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[41444]: osdmap e65: 8 total, 8 up, 8 in 2024-02-05T07:34:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:01 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:01 smithi050 ceph-mon[33492]: pgmap v196: 68 pgs: 14 active+clean, 9 creating+peering, 45 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-02-05T07:34:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:34:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-05T07:34:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:01 smithi050 ceph-mon[33492]: osdmap e65: 8 total, 8 up, 8 in 2024-02-05T07:34:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:01 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:01.790 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.50 --placement '1;smithi050=iscsi.a' 2024-02-05T07:34:02.049 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:34:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:02 smithi050 ceph-mon[33492]: osdmap e66: 8 total, 8 up, 8 in 2024-02-05T07:34:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[41444]: osdmap e66: 8 total, 8 up, 8 in 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[29879]: osdmap e66: 8 total, 8 up, 8 in 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-05T07:34:03.303 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:03 smithi050 ceph-mon[33492]: pgmap v199: 100 pgs: 19 active+clean, 14 creating+peering, 67 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-05T07:34:03.303 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:34:03.303 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:34:03.303 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:03 smithi050 ceph-mon[33492]: osdmap e67: 8 total, 8 up, 8 in 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[29879]: pgmap v199: 100 pgs: 19 active+clean, 14 creating+peering, 67 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[29879]: osdmap e67: 8 total, 8 up, 8 in 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[41444]: pgmap v199: 100 pgs: 19 active+clean, 14 creating+peering, 67 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-05T07:34:03.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:03 smithi043 ceph-mon[41444]: osdmap e67: 8 total, 8 up, 8 in 2024-02-05T07:34:03.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 -- 172.21.15.50:0/2456006968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc102f70 msgr2=0x7fecdc1033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:03.709 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 --2- 172.21.15.50:0/2456006968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc102f70 0x7fecdc1033d0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feccc0099c0 tx=0x7feccc009cd0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 -- 172.21.15.50:0/2456006968 shutdown_connections 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 --2- 172.21.15.50:0/2456006968 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fecdc103910 0x7fecdc10a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 --2- 172.21.15.50:0/2456006968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecdc101d70 0x7fecdc102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 --2- 172.21.15.50:0/2456006968 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc102f70 0x7fecdc1033d0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 -- 172.21.15.50:0/2456006968 >> 172.21.15.50:0/2456006968 conn(0x7fecdc0fd330 msgr2=0x7fecdc0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 -- 172.21.15.50:0/2456006968 shutdown_connections 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 -- 172.21.15.50:0/2456006968 wait complete. 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.706+0000 7fece3dbd700 1 Processor -- start 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece3dbd700 1 -- start start 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece3dbd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fecdc101d70 0x7fecdc075a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece3dbd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecdc102f70 0x7fecdc074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece3dbd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc103910 0x7fecdc074650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:03.710 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece3dbd700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc074df0 con 0x7fecdc102f70 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece3dbd700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc074f60 con 0x7fecdc103910 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece3dbd700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc1a1230 con 0x7fecdc101d70 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc103910 0x7fecdc074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece1358700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecdc102f70 0x7fecdc074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece1b59700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fecdc101d70 0x7fecdc075a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc103910 0x7fecdc074650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49324/0 (socket says 172.21.15.50:49324) 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 -- 172.21.15.50:0/122896458 learned_addr learned my addr 172.21.15.50:0/122896458 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 -- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fecdc101d70 msgr2=0x7fecdc075a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fecdc101d70 0x7fecdc075a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 -- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecdc102f70 msgr2=0x7fecdc074110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecdc102f70 0x7fecdc074110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 -- 172.21.15.50:0/122896458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feccc0095f0 con 0x7fecdc103910 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.707+0000 7fece235a700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc103910 0x7fecdc074650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007ee0 tx=0x7fecd400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.708+0000 7fecd2ffd700 1 -- 172.21.15.50:0/122896458 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4017070 con 0x7fecdc103910 2024-02-05T07:34:03.711 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.708+0000 7fecd2ffd700 1 -- 172.21.15.50:0/122896458 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fecd400f040 con 0x7fecdc103910 2024-02-05T07:34:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.708+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc1a14c0 con 0x7fecdc103910 2024-02-05T07:34:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.708+0000 7fecd2ffd700 1 -- 172.21.15.50:0/122896458 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd40054a0 con 0x7fecdc103910 2024-02-05T07:34:03.712 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.708+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc1a1a10 con 0x7fecdc103910 2024-02-05T07:34:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.709+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc108950 con 0x7fecdc103910 2024-02-05T07:34:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.709+0000 7fecd2ffd700 1 -- 172.21.15.50:0/122896458 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7fecd400db20 con 0x7fecdc103910 2024-02-05T07:34:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.709+0000 7fecd2ffd700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fecc806da70 0x7fecc806ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.710+0000 7fecd2ffd700 1 -- 172.21.15.50:0/122896458 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 7121+0+0 (secure 0 0 0) 0x7fecd408ccc0 con 0x7fecdc103910 2024-02-05T07:34:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.710+0000 7fece1b59700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fecc806da70 0x7fecc806ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:03.713 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.710+0000 7fece1b59700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fecc806da70 0x7fecc806ff30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fecdc102dd0 tx=0x7fecd800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:03.716 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.714+0000 7fecd2ffd700 1 -- 172.21.15.50:0/122896458 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fecd40562d0 con 0x7fecdc103910 2024-02-05T07:34:03.937 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.935+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.50", "placement": "1;smithi050=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fecdc062160 con 0x7fecc806da70 2024-02-05T07:34:03.945 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.943+0000 7fecd2ffd700 1 -- 172.21.15.50:0/122896458 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7fecdc062160 con 0x7fecc806da70 2024-02-05T07:34:03.945 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled iscsi.datapool update... 2024-02-05T07:34:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.945+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fecc806da70 msgr2=0x7fecc806ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.945+0000 7fece3dbd700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fecc806da70 0x7fecc806ff30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fecdc102dd0 tx=0x7fecd800b410 comp rx=0 tx=0).stop 2024-02-05T07:34:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.945+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc103910 msgr2=0x7fecdc074650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.945+0000 7fece3dbd700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc103910 0x7fecdc074650 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007ee0 tx=0x7fecd400d3b0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 shutdown_connections 2024-02-05T07:34:03.947 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fecc806da70 0x7fecc806ff30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fecdc103910 0x7fecdc074650 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fecdc101d70 0x7fecdc075a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 --2- 172.21.15.50:0/122896458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fecdc102f70 0x7fecdc074110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 >> 172.21.15.50:0/122896458 conn(0x7fecdc0fd330 msgr2=0x7fecdc0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 shutdown_connections 2024-02-05T07:34:03.948 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:03.946+0000 7fece3dbd700 1 -- 172.21.15.50:0/122896458 wait complete. 2024-02-05T07:34:04.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:04 smithi050 ceph-mon[33492]: osdmap e68: 8 total, 8 up, 8 in 2024-02-05T07:34:04.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.248 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.248 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:04 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[29879]: osdmap e68: 8 total, 8 up, 8 in 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[41444]: osdmap e68: 8 total, 8 up, 8 in 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-05T07:34:04.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:04 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:04.569 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-02-05T07:34:04.569 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:34:04.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-05T07:34:04.605 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:34:04.605 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-02-05T07:34:04.636 DEBUG:teuthology.orchestra.run.smithi050:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@iscsi.iscsi.a.service 2024-02-05T07:34:04.679 INFO:tasks.cephadm:Adding prometheus.a on smithi050 2024-02-05T07:34:04.679 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply prometheus '1;smithi050=a' 2024-02-05T07:34:04.956 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:34:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: pgmap v202: 132 pgs: 65 active+clean, 14 creating+peering, 53 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:34:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.50", "placement": "1;smithi050=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: Saving service iscsi.datapool spec with placement smithi050=iscsi.a;count:1 2024-02-05T07:34:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:34:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:34:05.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: osdmap e69: 8 total, 8 up, 8 in 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[29879]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: pgmap v202: 132 pgs: 65 active+clean, 14 creating+peering, 53 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.50", "placement": "1;smithi050=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: Saving service iscsi.datapool spec with placement smithi050=iscsi.a;count:1 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: osdmap e69: 8 total, 8 up, 8 in 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:05 smithi043 ceph-mon[41444]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:34:05.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: pgmap v202: 132 pgs: 65 active+clean, 14 creating+peering, 53 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: from='client.24394 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.50", "placement": "1;smithi050=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: Saving service iscsi.datapool spec with placement smithi050=iscsi.a;count:1 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: osdmap e69: 8 total, 8 up, 8 in 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2658810356' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-05T07:34:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:05 smithi050 ceph-mon[33492]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:34:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:34:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:34:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: osdmap e70: 8 total, 8 up, 8 in 2024-02-05T07:34:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: osdmap e70: 8 total, 8 up, 8 in 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-05T07:34:06.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:06 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:34:06.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3648991631' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: osdmap e70: 8 total, 8 up, 8 in 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-05T07:34:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:06 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:34:06.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 -- 172.21.15.50:0/3355959188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 msgr2=0x7f3328074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:06.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 --2- 172.21.15.50:0/3355959188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 0x7f3328074130 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f331800b0a0 tx=0x7f331800b3b0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.432 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 -- 172.21.15.50:0/3355959188 shutdown_connections 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 --2- 172.21.15.50:0/3355959188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3328109c50 0x7f332810c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 --2- 172.21.15.50:0/3355959188 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 0x7f3328074130 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 --2- 172.21.15.50:0/3355959188 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 0x7f3328074b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 -- 172.21.15.50:0/3355959188 >> 172.21.15.50:0/3355959188 conn(0x7f33280fd330 msgr2=0x7f33280ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 -- 172.21.15.50:0/3355959188 shutdown_connections 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 -- 172.21.15.50:0/3355959188 wait complete. 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.429+0000 7f332d777700 1 Processor -- start 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f332d777700 1 -- start start 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f332d777700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 0x7f33281995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f332d777700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 0x7f3328199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f332d777700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3328109c50 0x7f33281a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f332d777700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33281a1170 con 0x7f3328109c50 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f332d777700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33281a12b0 con 0x7f3328075cd0 2024-02-05T07:34:06.433 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f332d777700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33281a1440 con 0x7f3328074700 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f33267fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 0x7f3328199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 0x7f33281995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 0x7f33281995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.50:55376/0 (socket says 172.21.15.50:55376) 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f33267fc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 0x7f3328199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49346/0 (socket says 172.21.15.50:49346) 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 -- 172.21.15.50:0/2551301318 learned_addr learned my addr 172.21.15.50:0/2551301318 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f33277fe700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3328109c50 0x7f33281a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 -- 172.21.15.50:0/2551301318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 msgr2=0x7f3328199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 0x7f3328199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 -- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3328109c50 msgr2=0x7f33281a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3328109c50 0x7f33281a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 -- 172.21.15.50:0/2551301318 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3318009d00 con 0x7f3328074700 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.430+0000 7f3326ffd700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 0x7f33281995e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33180078c0 tx=0x7f331800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.431+0000 7f331ffff700 1 -- 172.21.15.50:0/2551301318 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331800e040 con 0x7f3328074700 2024-02-05T07:34:06.434 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.431+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33281a16c0 con 0x7f3328074700 2024-02-05T07:34:06.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.431+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33281a1b30 con 0x7f3328074700 2024-02-05T07:34:06.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.431+0000 7f331ffff700 1 -- 172.21.15.50:0/2551301318 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3318003c00 con 0x7f3328074700 2024-02-05T07:34:06.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.431+0000 7f331ffff700 1 -- 172.21.15.50:0/2551301318 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331801b460 con 0x7f3328074700 2024-02-05T07:34:06.435 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.432+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3308005120 con 0x7f3328074700 2024-02-05T07:34:06.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.432+0000 7f331ffff700 1 -- 172.21.15.50:0/2551301318 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7f331801d470 con 0x7f3328074700 2024-02-05T07:34:06.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.433+0000 7f331ffff700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f331406d9d0 0x7f331406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:06.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.433+0000 7f331ffff700 1 -- 172.21.15.50:0/2551301318 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f331808e5d0 con 0x7f3328074700 2024-02-05T07:34:06.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.433+0000 7f33267fc700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f331406d9d0 0x7f331406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:06.436 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.433+0000 7f33267fc700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f331406d9d0 0x7f331406fe90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3310009c60 tx=0x7f3310009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:06.443 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.442+0000 7f331ffff700 1 -- 172.21.15.50:0/2551301318 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3318057b30 con 0x7f3328074700 2024-02-05T07:34:06.665 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.663+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f3308000bc0 con 0x7f331406d9d0 2024-02-05T07:34:06.671 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.669+0000 7f331ffff700 1 -- 172.21.15.50:0/2551301318 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f3308000bc0 con 0x7f331406d9d0 2024-02-05T07:34:06.671 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled prometheus update... 2024-02-05T07:34:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f331406d9d0 msgr2=0x7f331406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f331406d9d0 0x7f331406fe90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3310009c60 tx=0x7f3310009260 comp rx=0 tx=0).stop 2024-02-05T07:34:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 msgr2=0x7f33281995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 0x7f33281995e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f33180078c0 tx=0x7f331800bab0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 shutdown_connections 2024-02-05T07:34:06.673 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f331406d9d0 0x7f331406fe90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3328109c50 0x7f33281a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f3328074700 0x7f33281995e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.671+0000 7f332d777700 1 --2- 172.21.15.50:0/2551301318 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f3328075cd0 0x7f3328199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:06.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.672+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 >> 172.21.15.50:0/2551301318 conn(0x7f33280fd330 msgr2=0x7f33280ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:06.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.672+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 shutdown_connections 2024-02-05T07:34:06.674 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:06.672+0000 7f332d777700 1 -- 172.21.15.50:0/2551301318 wait complete. 2024-02-05T07:34:07.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[29879]: pgmap v205: 132 pgs: 89 active+clean, 20 creating+peering, 23 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:34:07.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[29879]: Deploying daemon iscsi.iscsi.a on smithi050 2024-02-05T07:34:07.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[29879]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-05T07:34:07.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[29879]: Cluster is now healthy 2024-02-05T07:34:07.291 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:07.291 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[41444]: pgmap v205: 132 pgs: 89 active+clean, 20 creating+peering, 23 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:34:07.291 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[41444]: Deploying daemon iscsi.iscsi.a on smithi050 2024-02-05T07:34:07.292 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[41444]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-05T07:34:07.292 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[41444]: Cluster is now healthy 2024-02-05T07:34:07.292 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:07 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:07.317 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:07 smithi050 ceph-mon[33492]: pgmap v205: 132 pgs: 89 active+clean, 20 creating+peering, 23 unknown; 578 KiB data, 238 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:34:07.317 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:07 smithi050 ceph-mon[33492]: Deploying daemon iscsi.iscsi.a on smithi050 2024-02-05T07:34:07.317 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:07 smithi050 ceph-mon[33492]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-05T07:34:07.318 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:07 smithi050 ceph-mon[33492]: Cluster is now healthy 2024-02-05T07:34:07.318 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:07 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:07.364 DEBUG:teuthology.orchestra.run.smithi050:prometheus.a> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@prometheus.a.service 2024-02-05T07:34:07.367 INFO:tasks.cephadm:Adding node-exporter.a on smithi043 2024-02-05T07:34:07.367 INFO:tasks.cephadm:Adding node-exporter.b on smithi050 2024-02-05T07:34:07.367 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply node-exporter '2;smithi043=a;smithi050=b' 2024-02-05T07:34:07.778 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:34:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:08 smithi043 ceph-mon[29879]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:08 smithi043 ceph-mon[29879]: Saving service prometheus spec with placement smithi050=a;count:1 2024-02-05T07:34:08.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:08 smithi043 ceph-mon[41444]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:08.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:08 smithi043 ceph-mon[41444]: Saving service prometheus spec with placement smithi050=a;count:1 2024-02-05T07:34:08.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:08 smithi050 ceph-mon[33492]: from='client.24416 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:08.314 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:08 smithi050 ceph-mon[33492]: Saving service prometheus spec with placement smithi050=a;count:1 2024-02-05T07:34:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:09 smithi043 ceph-mon[29879]: pgmap v206: 132 pgs: 114 active+clean, 11 creating+peering, 7 unknown; 579 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.6 KiB/s wr, 79 op/s 2024-02-05T07:34:09.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:09 smithi043 ceph-mon[41444]: pgmap v206: 132 pgs: 114 active+clean, 11 creating+peering, 7 unknown; 579 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.6 KiB/s wr, 79 op/s 2024-02-05T07:34:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:09 smithi050 ceph-mon[33492]: pgmap v206: 132 pgs: 114 active+clean, 11 creating+peering, 7 unknown; 579 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 2.6 KiB/s wr, 79 op/s 2024-02-05T07:34:09.983 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:09 smithi050 podman[63066]: 2024-02-05 07:34:09.850168709 +0000 UTC m=+0.370442064 container create 8bf0cb917394ff82796509dec6a9da803341477e4de7810012b5bf93980d7895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T07:34:10.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 podman[63066]: 2024-02-05 07:34:10.178885231 +0000 UTC m=+0.699158599 container init 8bf0cb917394ff82796509dec6a9da803341477e4de7810012b5bf93980d7895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-05T07:34:10.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 podman[63066]: 2024-02-05 07:34:10.185001496 +0000 UTC m=+0.705274850 container start 8bf0cb917394ff82796509dec6a9da803341477e4de7810012b5bf93980d7895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-05T07:34:10.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 bash[63066]: 8bf0cb917394ff82796509dec6a9da803341477e4de7810012b5bf93980d7895 2024-02-05T07:34:10.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 systemd[1]: Started Ceph iscsi.iscsi.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.170+0000 7fa89b848700 1 -- 172.21.15.50:0/3263062720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 msgr2=0x7fa8941033e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.170+0000 7fa89b848700 1 --2- 172.21.15.50:0/3263062720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa8941033e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8840099c0 tx=0x7fa884009cd0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.170+0000 7fa89b848700 1 -- 172.21.15.50:0/3263062720 shutdown_connections 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.170+0000 7fa89b848700 1 --2- 172.21.15.50:0/3263062720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa894103920 0x7fa89410a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.170+0000 7fa89b848700 1 --2- 172.21.15.50:0/3263062720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa894101d80 0x7fa8941021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.170+0000 7fa89b848700 1 --2- 172.21.15.50:0/3263062720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa8941033e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.170+0000 7fa89b848700 1 -- 172.21.15.50:0/3263062720 >> 172.21.15.50:0/3263062720 conn(0x7fa8940fd340 msgr2=0x7fa8940ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 -- 172.21.15.50:0/3263062720 shutdown_connections 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 -- 172.21.15.50:0/3263062720 wait complete. 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 Processor -- start 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 -- start start 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa894101d80 0x7fa8941971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:10.358 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa894197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa894103920 0x7fa89419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419ed50 con 0x7fa894103920 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419ee90 con 0x7fa894102f80 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa89b848700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89419f020 con 0x7fa894101d80 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa898de3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa894197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa898de3700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa894197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:49370/0 (socket says 172.21.15.50:49370) 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa898de3700 1 -- 172.21.15.50:0/1238116563 learned_addr learned my addr 172.21.15.50:0/1238116563 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.171+0000 7fa899de5700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa894103920 0x7fa89419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa8995e4700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa894101d80 0x7fa8941971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa898de3700 1 -- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa894101d80 msgr2=0x7fa8941971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa898de3700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa894101d80 0x7fa8941971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa898de3700 1 -- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa894103920 msgr2=0x7fa89419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa898de3700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa894103920 0x7fa89419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.359 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa898de3700 1 -- 172.21.15.50:0/1238116563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8840095f0 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa898de3700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa894197700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa884000c00 tx=0x7fa88400f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa88a7fc700 1 -- 172.21.15.50:0/1238116563 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88401d070 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa88a7fc700 1 -- 172.21.15.50:0/1238116563 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa88400bd30 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa88a7fc700 1 -- 172.21.15.50:0/1238116563 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884022bf0 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89419f2a0 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.172+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89419f790 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.173+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa894108960 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.174+0000 7fa88a7fc700 1 -- 172.21.15.50:0/1238116563 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 15) v1 ==== 91406+0+0 (secure 0 0 0) 0x7fa884022d50 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.174+0000 7fa88a7fc700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa88006d9d0 0x7fa88006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.174+0000 7fa88a7fc700 1 -- 172.21.15.50:0/1238116563 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7148+0+0 (secure 0 0 0) 0x7fa88408e800 con 0x7fa894102f80 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.174+0000 7fa8995e4700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa88006d9d0 0x7fa88006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:10.360 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.175+0000 7fa8995e4700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa88006d9d0 0x7fa88006fe90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa894102de0 tx=0x7fa89000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:10.361 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.179+0000 7fa88a7fc700 1 -- 172.21.15.50:0/1238116563 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa884057e00 con 0x7fa894102f80 2024-02-05T07:34:10.420 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.419+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi050=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fa89410cde0 con 0x7fa88006d9d0 2024-02-05T07:34:10.426 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.425+0000 7fa88a7fc700 1 -- 172.21.15.50:0/1238116563 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fa89410cde0 con 0x7fa88006d9d0 2024-02-05T07:34:10.426 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled node-exporter update... 2024-02-05T07:34:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa88006d9d0 msgr2=0x7fa88006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa88006d9d0 0x7fa88006fe90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa894102de0 tx=0x7fa89000a3b0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 msgr2=0x7fa894197700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:10.429 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa894197700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa884000c00 tx=0x7fa88400f710 comp rx=0 tx=0).stop 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 shutdown_connections 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fa88006d9d0 0x7fa88006fe90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa894103920 0x7fa89419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa894101d80 0x7fa8941971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.428+0000 7fa89b848700 1 --2- 172.21.15.50:0/1238116563 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa894102f80 0x7fa894197700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.429+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 >> 172.21.15.50:0/1238116563 conn(0x7fa8940fd340 msgr2=0x7fa8940fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.429+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 shutdown_connections 2024-02-05T07:34:10.430 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:10.429+0000 7fa89b848700 1 -- 172.21.15.50:0/1238116563 wait complete. 2024-02-05T07:34:10.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug Started the configuration object watcher 2024-02-05T07:34:10.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug Checking for config object changes every 1s 2024-02-05T07:34:10.694 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug Processing osd blocklist entries for this node 2024-02-05T07:34:11.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug Reading the configuration object to update local LIO configuration 2024-02-05T07:34:11.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug Configuration does not have an entry for this host(smithi050.front.sepia.ceph.com) - nothing to define to LIO 2024-02-05T07:34:11.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: pgmap v207: 132 pgs: 132 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 4.4 KiB/s wr, 150 op/s 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: osdmap e71: 8 total, 8 up, 8 in 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.026 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:11.027 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.027 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/3357504662' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: pgmap v207: 132 pgs: 132 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 4.4 KiB/s wr, 150 op/s 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: osdmap e71: 8 total, 8 up, 8 in 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/3357504662' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: pgmap v207: 132 pgs: 132 active+clean; 581 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 62 KiB/s rd, 4.4 KiB/s wr, 150 op/s 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: osdmap e71: 8 total, 8 up, 8 in 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:11.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:11.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/3357504662' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-05T07:34:11.552 DEBUG:teuthology.orchestra.run.smithi043:node-exporter.a> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.a.service 2024-02-05T07:34:11.554 DEBUG:teuthology.orchestra.run.smithi050:node-exporter.b> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.b.service 2024-02-05T07:34:11.557 INFO:tasks.cephadm:Adding alertmanager.a on smithi043 2024-02-05T07:34:11.557 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply alertmanager '1;smithi043=a' 2024-02-05T07:34:12.274 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:11 smithi050 ceph-mon[33492]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-05T07:34:12.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:11 smithi050 ceph-mon[33492]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi050=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:12.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:11 smithi050 ceph-mon[33492]: Saving service node-exporter spec with placement smithi043=a;smithi050=b;count:2 2024-02-05T07:34:12.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:11 smithi050 ceph-mon[33492]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-05T07:34:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[29879]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-05T07:34:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[29879]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi050=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[29879]: Saving service node-exporter spec with placement smithi043=a;smithi050=b;count:2 2024-02-05T07:34:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[29879]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-05T07:34:12.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[41444]: Checking pool "datapool" exists for service iscsi.datapool 2024-02-05T07:34:12.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[41444]: from='client.24421 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi050=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:12.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[41444]: Saving service node-exporter spec with placement smithi043=a;smithi050=b;count:2 2024-02-05T07:34:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:11 smithi043 ceph-mon[41444]: mgrmap e16: y(active, since 4m), standbys: x 2024-02-05T07:34:12.540 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:34:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:12 smithi043 ceph-mon[29879]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.0 KiB/s wr, 158 op/s 2024-02-05T07:34:13.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:12 smithi043 ceph-mon[41444]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.0 KiB/s wr, 158 op/s 2024-02-05T07:34:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:12 smithi050 ceph-mon[33492]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 65 KiB/s rd, 5.0 KiB/s wr, 158 op/s 2024-02-05T07:34:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 -- 172.21.15.50:0/990209139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4100e60 msgr2=0x7f43e41012e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 --2- 172.21.15.50:0/990209139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4100e60 0x7f43e41012e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009990 tx=0x7f43cc009ca0 comp rx=0 tx=0).stop 2024-02-05T07:34:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 -- 172.21.15.50:0/990209139 shutdown_connections 2024-02-05T07:34:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 --2- 172.21.15.50:0/990209139 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e4101820 0x7f43e41090c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:14.881 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 --2- 172.21.15.50:0/990209139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4102200 0x7f43e4102620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 --2- 172.21.15.50:0/990209139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4100e60 0x7f43e41012e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 -- 172.21.15.50:0/990209139 >> 172.21.15.50:0/990209139 conn(0x7f43e40fb990 msgr2=0x7f43e40fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 -- 172.21.15.50:0/990209139 shutdown_connections 2024-02-05T07:34:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.880+0000 7f43e9c16700 1 -- 172.21.15.50:0/990209139 wait complete. 2024-02-05T07:34:14.882 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 Processor -- start 2024-02-05T07:34:14.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 -- start start 2024-02-05T07:34:14.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4100e60 0x7f43e4197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:14.883 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4101820 0x7f43e41977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e4102200 0x7f43e419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e419eda0 con 0x7f43e4100e60 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e419eee0 con 0x7f43e4101820 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e9c16700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e419f070 con 0x7f43e4102200 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4100e60 0x7f43e4197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e2ffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4101820 0x7f43e41977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e37fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4100e60 0x7f43e4197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.50:57606/0 (socket says 172.21.15.50:57606) 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e37fe700 1 -- 172.21.15.50:0/1533547165 learned_addr learned my addr 172.21.15.50:0/1533547165 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.881+0000 7f43e3fff700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e4102200 0x7f43e419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e3fff700 1 -- 172.21.15.50:0/1533547165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4101820 msgr2=0x7f43e41977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e3fff700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4101820 0x7f43e41977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e3fff700 1 -- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4100e60 msgr2=0x7f43e4197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e3fff700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4100e60 0x7f43e4197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e3fff700 1 -- 172.21.15.50:0/1533547165 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43cc0095f0 con 0x7f43e4102200 2024-02-05T07:34:14.884 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e37fe700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4100e60 0x7f43e4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e2ffd700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4101820 0x7f43e41977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e3fff700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e4102200 0x7f43e419e7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43dc007f40 tx=0x7f43dc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e0ff9700 1 -- 172.21.15.50:0/1533547165 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc004db0 con 0x7f43e4102200 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e0ff9700 1 -- 172.21.15.50:0/1533547165 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f43dc00f040 con 0x7f43e4102200 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e419f350 con 0x7f43e4102200 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e0ff9700 1 -- 172.21.15.50:0/1533547165 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0057d0 con 0x7f43e4102200 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.882+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e419f800 con 0x7f43e4102200 2024-02-05T07:34:14.885 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.883+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4108a50 con 0x7f43e4102200 2024-02-05T07:34:14.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.883+0000 7f43e0ff9700 1 -- 172.21.15.50:0/1533547165 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 16) v1 ==== 91457+0+0 (secure 0 0 0) 0x7f43dc005930 con 0x7f43e4102200 2024-02-05T07:34:14.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.884+0000 7f43e0ff9700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f43d006da60 0x7f43d006ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:14.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.884+0000 7f43e0ff9700 1 -- 172.21.15.50:0/1533547165 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f43dc08d030 con 0x7f43e4102200 2024-02-05T07:34:14.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.884+0000 7f43e37fe700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f43d006da60 0x7f43d006ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:14.886 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.884+0000 7f43e37fe700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f43d006da60 0x7f43d006ff20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005950 tx=0x7f43d400b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:14.889 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:14.889+0000 7f43e0ff9700 1 -- 172.21.15.50:0/1533547165 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43dc0564b0 con 0x7f43e4102200 2024-02-05T07:34:15.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:14 smithi050 ceph-mon[33492]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 138 op/s 2024-02-05T07:34:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:34:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:34:15.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:14 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.107 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.106+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f43e410bc80 con 0x7f43d006da60 2024-02-05T07:34:15.113 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.112+0000 7f43e0ff9700 1 -- 172.21.15.50:0/1533547165 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f43e410bc80 con 0x7f43d006da60 2024-02-05T07:34:15.113 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled alertmanager update... 2024-02-05T07:34:15.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f43d006da60 msgr2=0x7f43d006ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:15.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f43d006da60 0x7f43d006ff20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005950 tx=0x7f43d400b500 comp rx=0 tx=0).stop 2024-02-05T07:34:15.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e4102200 msgr2=0x7f43e419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:15.115 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e4102200 0x7f43e419e7d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f43dc007f40 tx=0x7f43dc00d3b0 comp rx=0 tx=0).stop 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 shutdown_connections 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f43d006da60 0x7f43d006ff20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f43e4102200 0x7f43e419e7d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43e4100e60 0x7f43e4197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 --2- 172.21.15.50:0/1533547165 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f43e4101820 0x7f43e41977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 >> 172.21.15.50:0/1533547165 conn(0x7f43e40fb990 msgr2=0x7f43e40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 shutdown_connections 2024-02-05T07:34:15.116 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:15.115+0000 7f43e9c16700 1 -- 172.21.15.50:0/1533547165 wait complete. 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[41444]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 138 op/s 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[29879]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 138 op/s 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:34:15.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:34:15.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:14 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:15.722 DEBUG:teuthology.orchestra.run.smithi043:alertmanager.a> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@alertmanager.a.service 2024-02-05T07:34:15.725 INFO:tasks.cephadm:Adding grafana.a on smithi050 2024-02-05T07:34:15.726 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph orch apply grafana '1;smithi050=a' 2024-02-05T07:34:15.979 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:34:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[41444]: Deploying daemon prometheus.a on smithi050 2024-02-05T07:34:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-05T07:34:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-05T07:34:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-02-05T07:34:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[29879]: Deploying daemon prometheus.a on smithi050 2024-02-05T07:34:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-05T07:34:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-05T07:34:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-02-05T07:34:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:15 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:16.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:15 smithi050 ceph-mon[33492]: Deploying daemon prometheus.a on smithi050 2024-02-05T07:34:16.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:15 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-02-05T07:34:16.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:15 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-02-05T07:34:16.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:15 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]: dispatch 2024-02-05T07:34:16.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:15 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:16.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:15 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[29879]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-05T07:34:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[29879]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[29879]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-02-05T07:34:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-05T07:34:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-02-05T07:34:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-02-05T07:34:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[29879]: osdmap e72: 8 total, 8 up, 8 in 2024-02-05T07:34:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[41444]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-05T07:34:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[41444]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[41444]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-02-05T07:34:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-05T07:34:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-02-05T07:34:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-02-05T07:34:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:16 smithi043 ceph-mon[41444]: osdmap e72: 8 total, 8 up, 8 in 2024-02-05T07:34:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:16 smithi050 ceph-mon[33492]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-05T07:34:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:16 smithi050 ceph-mon[33492]: from='client.24455 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:16 smithi050 ceph-mon[33492]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-02-05T07:34:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:16 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-02-05T07:34:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:16 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-02-05T07:34:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:16 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 2]}]': finished 2024-02-05T07:34:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:16 smithi050 ceph-mon[33492]: osdmap e72: 8 total, 8 up, 8 in 2024-02-05T07:34:18.210 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:17 smithi050 ceph-mon[33492]: osdmap e73: 8 total, 8 up, 8 in 2024-02-05T07:34:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:17 smithi043 ceph-mon[29879]: osdmap e73: 8 total, 8 up, 8 in 2024-02-05T07:34:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:17 smithi043 ceph-mon[41444]: osdmap e73: 8 total, 8 up, 8 in 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 -- 172.21.15.50:0/913373890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a441030c0 msgr2=0x7f2a44103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/913373890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a441030c0 0x7f2a44103540 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009990 tx=0x7f2a38009ca0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 -- 172.21.15.50:0/913373890 shutdown_connections 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/913373890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a44103a80 0x7f2a4410a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/913373890 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a44100090 0x7f2a441004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/913373890 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a441030c0 0x7f2a44103540 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 -- 172.21.15.50:0/913373890 >> 172.21.15.50:0/913373890 conn(0x7f2a440fb9c0 msgr2=0x7f2a440fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 -- 172.21.15.50:0/913373890 shutdown_connections 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 -- 172.21.15.50:0/913373890 wait complete. 2024-02-05T07:34:18.378 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 Processor -- start 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.337+0000 7f2a4a88f700 1 -- start start 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a4a88f700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a44100090 0x7f2a44197270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a4a88f700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a441030c0 0x7f2a441977b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a4a88f700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a44103a80 0x7f2a4419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a4a88f700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419edb0 con 0x7f2a441030c0 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a4a88f700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419eef0 con 0x7f2a44103a80 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a4a88f700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419f080 con 0x7f2a44100090 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a44103a80 0x7f2a4419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a44103a80 0x7f2a4419e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:54396/0 (socket says 172.21.15.50:54396) 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 -- 172.21.15.50:0/2466908067 learned_addr learned my addr 172.21.15.50:0/2466908067 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a43fff700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a44100090 0x7f2a44197270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a437fe700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a441030c0 0x7f2a441977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 -- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a44100090 msgr2=0x7f2a44197270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:18.379 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a44100090 0x7f2a44197270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 -- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a441030c0 msgr2=0x7f2a441977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a441030c0 0x7f2a441977b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 -- 172.21.15.50:0/2466908067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a380095f0 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.338+0000 7f2a48e2c700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a44103a80 0x7f2a4419e7e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00ef30 tx=0x7f2a3c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.339+0000 7f2a417fa700 1 -- 172.21.15.50:0/2466908067 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c018070 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.339+0000 7f2a417fa700 1 -- 172.21.15.50:0/2466908067 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2a3c00f040 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.339+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4419f300 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.339+0000 7f2a417fa700 1 -- 172.21.15.50:0/2466908067 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c00cba0 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.339+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4419f850 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.340+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a44108aa0 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.340+0000 7f2a417fa700 1 -- 172.21.15.50:0/2466908067 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) v1 ==== 91457+0+0 (secure 0 0 0) 0x7f2a3c00cd00 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.341+0000 7f2a417fa700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2a2c06da60 0x7f2a2c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.341+0000 7f2a417fa700 1 -- 172.21.15.50:0/2466908067 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f2a3c08d1f0 con 0x7f2a44103a80 2024-02-05T07:34:18.380 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.341+0000 7f2a43fff700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2a2c06da60 0x7f2a2c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:18.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.342+0000 7f2a43fff700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2a2c06da60 0x7f2a2c06ff20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2a4406a1a0 tx=0x7f2a34009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:18.381 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.346+0000 7f2a417fa700 1 -- 172.21.15.50:0/2466908067 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a3c0566b0 con 0x7f2a44103a80 2024-02-05T07:34:18.568 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.567+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 --> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a4410cd90 con 0x7f2a2c06da60 2024-02-05T07:34:18.576 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.575+0000 7f2a417fa700 1 -- 172.21.15.50:0/2466908067 <== mgr.14152 v2:172.21.15.43:6800/629017799 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2a4410cd90 con 0x7f2a2c06da60 2024-02-05T07:34:18.576 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled grafana update... 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2a2c06da60 msgr2=0x7f2a2c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2a2c06da60 0x7f2a2c06ff20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2a4406a1a0 tx=0x7f2a34009260 comp rx=0 tx=0).stop 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a44103a80 msgr2=0x7f2a4419e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a44103a80 0x7f2a4419e7e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00ef30 tx=0x7f2a3c00c3a0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 shutdown_connections 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f2a2c06da60 0x7f2a2c06ff20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f2a44103a80 0x7f2a4419e7e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f2a44100090 0x7f2a44197270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 --2- 172.21.15.50:0/2466908067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a441030c0 0x7f2a441977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 >> 172.21.15.50:0/2466908067 conn(0x7f2a440fb9c0 msgr2=0x7f2a440fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:18.578 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 shutdown_connections 2024-02-05T07:34:18.579 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:18.577+0000 7f2a4a88f700 1 -- 172.21.15.50:0/2466908067 wait complete. 2024-02-05T07:34:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:18 smithi043 ceph-mon[29879]: pgmap v214: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 572 B/s rd, 143 B/s wr, 1 op/s 2024-02-05T07:34:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:18 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:18 smithi043 ceph-mon[41444]: pgmap v214: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 572 B/s rd, 143 B/s wr, 1 op/s 2024-02-05T07:34:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:18 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:18 smithi050 ceph-mon[33492]: pgmap v214: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 572 B/s rd, 143 B/s wr, 1 op/s 2024-02-05T07:34:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:18 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:19.562 DEBUG:teuthology.orchestra.run.smithi050:grafana.a> sudo journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@grafana.a.service 2024-02-05T07:34:19.564 INFO:tasks.cephadm:Setting up client nodes... 2024-02-05T07:34:19.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-05T07:34:19.820 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:20.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:19 smithi043 ceph-mon[29879]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:20.220 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:19 smithi043 ceph-mon[29879]: Saving service grafana spec with placement smithi050=a;count:1 2024-02-05T07:34:20.220 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:19 smithi043 ceph-mon[41444]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:20.220 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:19 smithi043 ceph-mon[41444]: Saving service grafana spec with placement smithi050=a;count:1 2024-02-05T07:34:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:19 smithi050 ceph-mon[33492]: from='client.24463 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-05T07:34:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:19 smithi050 ceph-mon[33492]: Saving service grafana spec with placement smithi050=a;count:1 2024-02-05T07:34:20.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:34:21.245 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:20 smithi043 ceph-mon[29879]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:34:21.246 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:20 smithi043 ceph-mon[41444]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:34:21.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:20 smithi050 ceph-mon[33492]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-02-05T07:34:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 -- 172.21.15.43:0/3515355320 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0105130 msgr2=0x7f14f0107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 --2- 172.21.15.43:0/3515355320 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0105130 0x7f14f0107520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009990 tx=0x7f14e4009ca0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 -- 172.21.15.43:0/3515355320 shutdown_connections 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 --2- 172.21.15.43:0/3515355320 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14f0107a60 0x7f14f0109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 --2- 172.21.15.43:0/3515355320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0102800 0x7f14f0104bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 --2- 172.21.15.43:0/3515355320 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0105130 0x7f14f0107520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 -- 172.21.15.43:0/3515355320 >> 172.21.15.43:0/3515355320 conn(0x7f14f00fc210 msgr2=0x7f14f00fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 -- 172.21.15.43:0/3515355320 shutdown_connections 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 -- 172.21.15.43:0/3515355320 wait complete. 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 Processor -- start 2024-02-05T07:34:21.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.630+0000 7f14f52be700 1 -- start start 2024-02-05T07:34:21.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14f52be700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14f0102800 0x7f14f01971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14f52be700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0105130 0x7f14f0197730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14f52be700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0107a60 0x7f14f019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14f52be700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f019ed30 con 0x7f14f0105130 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14f52be700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f019ee70 con 0x7f14f0102800 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14f52be700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f019f000 con 0x7f14f0107a60 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0105130 0x7f14f0197730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0105130 0x7f14f0197730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46100/0 (socket says 172.21.15.43:46100) 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14eeffd700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14f0102800 0x7f14f01971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ef7fe700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0107a60 0x7f14f019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 -- 172.21.15.43:0/2073106108 learned_addr learned my addr 172.21.15.43:0/2073106108 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 -- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0107a60 msgr2=0x7f14f019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0107a60 0x7f14f019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 -- 172.21.15.43:0/2073106108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14f0102800 msgr2=0x7f14f01971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:21.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14f0102800 0x7f14f01971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 -- 172.21.15.43:0/2073106108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e0009520 con 0x7f14f0105130 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ef7fe700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0107a60 0x7f14f019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14ee7fc700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0105130 0x7f14f0197730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f14e40059d0 tx=0x7f14e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.631+0000 7f14d7fff700 1 -- 172.21.15.43:0/2073106108 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e401d070 con 0x7f14f0105130 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.632+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e40095f0 con 0x7f14f0105130 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.632+0000 7f14d7fff700 1 -- 172.21.15.43:0/2073106108 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f14e400bdd0 con 0x7f14f0105130 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.632+0000 7f14d7fff700 1 -- 172.21.15.43:0/2073106108 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e4022bb0 con 0x7f14f0105130 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.632+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f019f500 con 0x7f14f0105130 2024-02-05T07:34:21.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.632+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f00fdd90 con 0x7f14f0105130 2024-02-05T07:34:21.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.633+0000 7f14d7fff700 1 -- 172.21.15.43:0/2073106108 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 16) v1 ==== 91457+0+0 (secure 0 0 0) 0x7f14e400b950 con 0x7f14f0105130 2024-02-05T07:34:21.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.633+0000 7f14d7fff700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f14d806db10 0x7f14d806ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:21.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.634+0000 7f14d7fff700 1 -- 172.21.15.43:0/2073106108 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f14e408ef70 con 0x7f14f0105130 2024-02-05T07:34:21.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.634+0000 7f14eeffd700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f14d806db10 0x7f14d806ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:21.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.634+0000 7f14eeffd700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f14d806db10 0x7f14d806ffd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009df0 tx=0x7f14e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:21.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.639+0000 7f14d7fff700 1 -- 172.21.15.43:0/2073106108 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f14e40c2d20 con 0x7f14f0105130 2024-02-05T07:34:21.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.924+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f14f010cb40 con 0x7f14f0105130 2024-02-05T07:34:21.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.926+0000 7f14d7fff700 1 -- 172.21.15.43:0/2073106108 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f14e4027070 con 0x7f14f0105130 2024-02-05T07:34:21.928 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-02-05T07:34:21.928 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQB9j8BlZrg3NxAAKkwr+Zz7LLGoH0Qs+Exhsw== 2024-02-05T07:34:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f14d806db10 msgr2=0x7f14d806ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f14d806db10 0x7f14d806ffd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f14e0009df0 tx=0x7f14e0009260 comp rx=0 tx=0).stop 2024-02-05T07:34:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0105130 msgr2=0x7f14f0197730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0105130 0x7f14f0197730 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f14e40059d0 tx=0x7f14e4004730 comp rx=0 tx=0).stop 2024-02-05T07:34:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 shutdown_connections 2024-02-05T07:34:21.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f14d806db10 0x7f14d806ffd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f14f0107a60 0x7f14f019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f14f0102800 0x7f14f01971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 --2- 172.21.15.43:0/2073106108 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14f0105130 0x7f14f0197730 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 >> 172.21.15.43:0/2073106108 conn(0x7f14f00fc210 msgr2=0x7f14f0103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 shutdown_connections 2024-02-05T07:34:21.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:21.928+0000 7f14f52be700 1 -- 172.21.15.43:0/2073106108 wait complete. 2024-02-05T07:34:22.263 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:22.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2073106108' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:22.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2073106108' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:34:22.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:22.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2073106108' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:22.264 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2073106108' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:34:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2073106108' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2073106108' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:34:22.660 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:34:22.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-05T07:34:22.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-05T07:34:22.704 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-05T07:34:23.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:22 smithi050 ceph-mon[33492]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 127 B/s wr, 2 op/s; 41 B/s, 1 objects/s recovering 2024-02-05T07:34:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:22 smithi043 ceph-mon[29879]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 127 B/s wr, 2 op/s; 41 B/s, 1 objects/s recovering 2024-02-05T07:34:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:22 smithi043 ceph-mon[41444]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 127 B/s wr, 2 op/s; 41 B/s, 1 objects/s recovering 2024-02-05T07:34:23.350 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.b/config 2024-02-05T07:34:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:24 smithi043 ceph-mon[29879]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-02-05T07:34:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:24 smithi043 ceph-mon[41444]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-02-05T07:34:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:24 smithi050 ceph-mon[33492]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-02-05T07:34:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:26 smithi043 ceph-mon[29879]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s; 48 B/s, 1 objects/s recovering 2024-02-05T07:34:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:26 smithi043 ceph-mon[41444]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s; 48 B/s, 1 objects/s recovering 2024-02-05T07:34:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:26 smithi050 ceph-mon[33492]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s; 48 B/s, 1 objects/s recovering 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.022+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/3515202365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d08107a50 msgr2=0x7f7d08109ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.022+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/3515202365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d08107a50 0x7f7d08109ec0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b0a0 tx=0x7f7d0000b3b0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.022+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/3515202365 shutdown_connections 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.022+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/3515202365 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d08107a50 0x7f7d08109ec0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.022+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/3515202365 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d081027f0 0x7f7d08104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.022+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/3515202365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d08105120 0x7f7d08107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.022+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/3515202365 >> 172.21.15.50:0/3515202365 conn(0x7f7d080fc220 msgr2=0x7f7d080fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.023+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/3515202365 shutdown_connections 2024-02-05T07:34:28.735 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.023+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/3515202365 wait complete. 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.023+0000 7f7d0d1bc700 1 Processor -- start 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0d1bc700 1 -- start start 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0d1bc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d081027f0 0x7f7d08197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0d1bc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d08105120 0x7f7d08197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0d1bc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d08107a50 0x7f7d0819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0d1bc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0819ede0 con 0x7f7d08107a50 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0d1bc700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0819ef20 con 0x7f7d081027f0 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0d1bc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0819f0b0 con 0x7f7d08105120 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d081027f0 0x7f7d08197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d081027f0 0x7f7d08197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.50:50844/0 (socket says 172.21.15.50:50844) 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 -- 172.21.15.50:0/746190474 learned_addr learned my addr 172.21.15.50:0/746190474 (peer_addr_for_me v2:172.21.15.50:0/0) 2024-02-05T07:34:28.736 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0659c700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d08105120 0x7f7d08197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d0759e700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d08107a50 0x7f7d0819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 -- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d08105120 msgr2=0x7f7d08197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d08105120 0x7f7d08197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 -- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d08107a50 msgr2=0x7f7d0819e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d08107a50 0x7f7d0819e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 -- 172.21.15.50:0/746190474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009d00 con 0x7f7d081027f0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.024+0000 7f7d06d9d700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d081027f0 0x7f7d08197250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800e9a0 tx=0x7f7cf800ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.025+0000 7f7ceffff700 1 -- 172.21.15.50:0/746190474 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8009790 con 0x7f7d081027f0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.025+0000 7f7ceffff700 1 -- 172.21.15.50:0/746190474 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7cf800cbd0 con 0x7f7d081027f0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.025+0000 7f7ceffff700 1 -- 172.21.15.50:0/746190474 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8018680 con 0x7f7d081027f0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.025+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0819f390 con 0x7f7d081027f0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.025+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0819f8e0 con 0x7f7d081027f0 2024-02-05T07:34:28.737 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.026+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d080fdda0 con 0x7f7d081027f0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.026+0000 7f7ceffff700 1 -- 172.21.15.50:0/746190474 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 16) v1 ==== 91457+0+0 (secure 0 0 0) 0x7f7cf8010430 con 0x7f7d081027f0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.027+0000 7f7ceffff700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7cf006db10 0x7f7cf006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.027+0000 7f7ceffff700 1 -- 172.21.15.50:0/746190474 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f7cf8014070 con 0x7f7d081027f0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.027+0000 7f7d0659c700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7cf006db10 0x7f7cf006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.027+0000 7f7d0659c700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7cf006db10 0x7f7cf006ffd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005fd0 tx=0x7f7cfc005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.031+0000 7f7ceffff700 1 -- 172.21.15.50:0/746190474 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cf8052d20 con 0x7f7d081027f0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.320+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f7d0810cdc0 con 0x7f7d081027f0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.325+0000 7f7ceffff700 1 -- 172.21.15.50:0/746190474 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f7cf8056e90 con 0x7f7d081027f0 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7cf006db10 msgr2=0x7f7cf006ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7cf006db10 0x7f7cf006ffd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc005fd0 tx=0x7f7cfc005f60 comp rx=0 tx=0).stop 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d081027f0 msgr2=0x7f7d08197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d081027f0 0x7f7d08197250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7cf800e9a0 tx=0x7f7cf800ed60 comp rx=0 tx=0).stop 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 shutdown_connections 2024-02-05T07:34:28.738 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f7cf006db10 0x7f7cf006ffd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d08107a50 0x7f7d0819e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d081027f0 0x7f7d08197250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 --2- 172.21.15.50:0/746190474 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d08105120 0x7f7d08197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.327+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 >> 172.21.15.50:0/746190474 conn(0x7f7d080fc220 msgr2=0x7f7d08103460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.328+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 shutdown_connections 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stderr:2024-02-05T07:34:28.328+0000 7f7d0d1bc700 1 -- 172.21.15.50:0/746190474 wait complete. 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2024-02-05T07:34:28.739 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQCEj8Bl5mNBExAAtzly9ga3/iesHAkU/2oxsw== 2024-02-05T07:34:29.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:28 smithi050 ceph-mon[33492]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 43 B/s, 1 objects/s recovering 2024-02-05T07:34:29.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.50:0/746190474' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:29.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:29.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[41444]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 43 B/s, 1 objects/s recovering 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.50:0/746190474' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[29879]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 908 B/s rd, 0 op/s; 43 B/s, 1 objects/s recovering 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.50:0/746190474' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-05T07:34:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-05T07:34:29.417 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-02-05T07:34:29.418 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-05T07:34:29.418 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-05T07:34:29.463 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-05T07:34:29.463 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-05T07:34:29.464 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph mgr dump --format=json 2024-02-05T07:34:29.718 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:30.505 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 systemd[1]: Starting Ceph prometheus.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:34:30.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:34:30.823 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 podman[64893]: 2024-02-05 07:34:30.515407357 +0000 UTC m=+0.019478406 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-05T07:34:30.823 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 podman[64893]: 2024-02-05 07:34:30.636605569 +0000 UTC m=+0.140676620 container create b2ee6c55bd7fd69b9d600bc7a65e5287a388670a58f340b793232964fb7680ff (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:34:31.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:30 smithi043 ceph-mon[29879]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-02-05T07:34:31.277 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:30 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:31.277 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:30 smithi043 ceph-mon[41444]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-02-05T07:34:31.278 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:30 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:31.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-mon[33492]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 36 B/s, 1 objects/s recovering 2024-02-05T07:34:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:31.323 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 podman[64893]: 2024-02-05 07:34:30.856280273 +0000 UTC m=+0.360351339 container init b2ee6c55bd7fd69b9d600bc7a65e5287a388670a58f340b793232964fb7680ff (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:34:31.323 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 podman[64893]: 2024-02-05 07:34:30.861720641 +0000 UTC m=+0.365791692 container start b2ee6c55bd7fd69b9d600bc7a65e5287a388670a58f340b793232964fb7680ff (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:34:31.323 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.885Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-05T07:34:31.323 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.885Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-05T07:34:31.323 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.885Z caller=main.go:570 level=info host_details="(Linux 5.14.0-416.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024 x86_64 smithi050 (none))" 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.885Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.885Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.886Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.886Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.887Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.887Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.889Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.889Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.244µs 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.889Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.890Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.890Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=55.392µs wal_replay_duration=505.297µs wbl_replay_duration=126ns total_replay_duration=582.789µs 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.891Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.891Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-05T07:34:31.324 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.891Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-05T07:34:31.325 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.907Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.334851ms db_storage=777ns remote_storage=1.033µs web_handler=375ns query_engine=542ns scrape=464.136µs scrape_sd=70.397µs notify=686ns notify_sd=1.431µs rules=15.624666ms tracing=5.913µs 2024-02-05T07:34:31.325 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.907Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-05T07:34:31.325 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:34:30.907Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-05T07:34:31.325 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 bash[64893]: b2ee6c55bd7fd69b9d600bc7a65e5287a388670a58f340b793232964fb7680ff 2024-02-05T07:34:31.325 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:34:30 smithi050 systemd[1]: Started Ceph prometheus.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 -- 172.21.15.43:0/2475857082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740699d0 msgr2=0x7f1f74069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 --2- 172.21.15.43:0/2475857082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740699d0 0x7f1f74069e30 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009920 tx=0x7f1f68009c30 comp rx=0 tx=0).stop 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 -- 172.21.15.43:0/2475857082 shutdown_connections 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 --2- 172.21.15.43:0/2475857082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f7406a370 0x7f1f7410ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 --2- 172.21.15.43:0/2475857082 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f74107d70 0x7f1f74108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 --2- 172.21.15.43:0/2475857082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f740699d0 0x7f1f74069e30 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 -- 172.21.15.43:0/2475857082 >> 172.21.15.43:0/2475857082 conn(0x7f1f74077b50 msgr2=0x7f1f74077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 -- 172.21.15.43:0/2475857082 shutdown_connections 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.523+0000 7f1f7b420700 1 -- 172.21.15.43:0/2475857082 wait complete. 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 Processor -- start 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 -- start start 2024-02-05T07:34:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f740699d0 0x7f1f74100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7406a370 0x7f1f74101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f74107d70 0x7f1f74101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741a5c00 con 0x7f1f7406a370 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741a5d70 con 0x7f1f740699d0 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f7b420700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741a5f30 con 0x7f1f74107d70 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f789bb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7406a370 0x7f1f74101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f789bb700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7406a370 0x7f1f74101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41872/0 (socket says 172.21.15.43:41872) 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f799bd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f74107d70 0x7f1f74101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f789bb700 1 -- 172.21.15.43:0/3526713956 learned_addr learned my addr 172.21.15.43:0/3526713956 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f791bc700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f740699d0 0x7f1f74100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f799bd700 1 -- 172.21.15.43:0/3526713956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f740699d0 msgr2=0x7f1f74100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f799bd700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f740699d0 0x7f1f74100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f799bd700 1 -- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7406a370 msgr2=0x7f1f74101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f799bd700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7406a370 0x7f1f74101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.524+0000 7f1f799bd700 1 -- 172.21.15.43:0/3526713956 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c009d00 con 0x7f1f74107d70 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.525+0000 7f1f799bd700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f74107d70 0x7f1f74101770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400c7a0 tx=0x7f1f6400cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.525+0000 7f1f727fc700 1 -- 172.21.15.43:0/3526713956 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64004a30 con 0x7f1f74107d70 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.525+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f68009520 con 0x7f1f74107d70 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.525+0000 7f1f727fc700 1 -- 172.21.15.43:0/3526713956 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1f640054a0 con 0x7f1f74107d70 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.525+0000 7f1f727fc700 1 -- 172.21.15.43:0/3526713956 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6400ea60 con 0x7f1f74107d70 2024-02-05T07:34:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.525+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f741a6530 con 0x7f1f74107d70 2024-02-05T07:34:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.526+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7410a180 con 0x7f1f74107d70 2024-02-05T07:34:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.526+0000 7f1f727fc700 1 -- 172.21.15.43:0/3526713956 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 16) v1 ==== 91457+0+0 (secure 0 0 0) 0x7f1f64005020 con 0x7f1f74107d70 2024-02-05T07:34:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.527+0000 7f1f727fc700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1f5c06da60 0x7f1f5c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.527+0000 7f1f791bc700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1f5c06da60 0x7f1f5c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.527+0000 7f1f727fc700 1 -- 172.21.15.43:0/3526713956 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f1f64013070 con 0x7f1f74107d70 2024-02-05T07:34:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.527+0000 7f1f791bc700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1f5c06da60 0x7f1f5c06ff20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009cd0 tx=0x7f1f6c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.531+0000 7f1f727fc700 1 -- 172.21.15.43:0/3526713956 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f64056800 con 0x7f1f74107d70 2024-02-05T07:34:31.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.793+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1f741021d0 con 0x7f1f74107d70 2024-02-05T07:34:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.797+0000 7f1f727fc700 1 -- 172.21.15.43:0/3526713956 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+174857 (secure 0 0 0) 0x7f1f64059ec0 con 0x7f1f74107d70 2024-02-05T07:34:31.800 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1f5c06da60 msgr2=0x7f1f5c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1f5c06da60 0x7f1f5c06ff20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009cd0 tx=0x7f1f6c00b040 comp rx=0 tx=0).stop 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f74107d70 msgr2=0x7f1f74101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f74107d70 0x7f1f74101770 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400c7a0 tx=0x7f1f6400cab0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 shutdown_connections 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f1f5c06da60 0x7f1f5c06ff20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1f74107d70 0x7f1f74101770 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1f740699d0 0x7f1f74100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 --2- 172.21.15.43:0/3526713956 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f7406a370 0x7f1f74101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 >> 172.21.15.43:0/3526713956 conn(0x7f1f74077b50 msgr2=0x7f1f740ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 shutdown_connections 2024-02-05T07:34:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:31.799+0000 7f1f7b420700 1 -- 172.21.15.43:0/3526713956 wait complete. 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3526713956' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T07:34:32.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3526713956' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T07:34:32.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3526713956' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: ignoring --setuser ceph since I am not root 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: ignoring --setgroup ceph since I am not root 2024-02-05T07:34:32.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-mgr[35623]: -- 172.21.15.50:0/2577993602 <== mon.1 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5598d93a5520 con 0x5598d937ac00 2024-02-05T07:34:32.479 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ignoring --setuser ceph since I am not root 2024-02-05T07:34:32.479 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ignoring --setgroup ceph since I am not root 2024-02-05T07:34:32.507 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":629017799},{"type":"v1","addr":"172.21.15.43:6801","nonce":629017799}]},"active_addr":"172.21.15.43:6801/629017799","active_change":"2024-02-05T07:29:15.032048+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24118,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1659834459}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2892606979}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3498131848}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":3646229614}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":1091771556}]}]} 2024-02-05T07:34:32.510 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-05T07:34:32.511 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-05T07:34:32.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T07:34:32.743 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:32.638+0000 7f0e94a7e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:34:32.760 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:33.002 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:32 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:32.741+0000 7f0e94a7e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:34:33.050 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:32.628+0000 7fa406c96200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-05T07:34:33.050 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:32.723+0000 7fa406c96200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-05T07:34:33.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-mon[29879]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T07:34:33.266 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-mon[29879]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-05T07:34:33.266 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-mon[41444]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T07:34:33.266 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-mon[41444]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-05T07:34:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:33 smithi050 ceph-mon[33492]: from='mgr.14152 172.21.15.43:0/471370154' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-05T07:34:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:33 smithi050 ceph-mon[33492]: mgrmap e17: y(active, since 5m), standbys: x 2024-02-05T07:34:33.520 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:33.421+0000 7f0e94a7e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:34:33.520 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:33.519+0000 7f0e94a7e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:34:33.717 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:33 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:33.379+0000 7fa406c96200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-05T07:34:33.717 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:33 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:33.473+0000 7fa406c96200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-05T07:34:33.717 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:33 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:33.617+0000 7fa406c96200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:34:33.717 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:33 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:33.715+0000 7fa406c96200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:34:33.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:33.664+0000 7f0e94a7e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-05T07:34:33.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:33.763+0000 7f0e94a7e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-05T07:34:34.072 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:33 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:33.801+0000 7fa406c96200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:34:34.072 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:34 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:34.071+0000 7fa406c96200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:34:34.127 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:33 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:33.849+0000 7f0e94a7e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-05T07:34:34.370 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:34 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:34.183+0000 7fa406c96200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:34:34.370 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:34 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:34.271+0000 7fa406c96200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:34:34.370 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:34 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:34.368+0000 7fa406c96200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:34:34.381 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:34.125+0000 7f0e94a7e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-05T07:34:34.381 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:34.245+0000 7f0e94a7e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-05T07:34:34.381 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:34.335+0000 7f0e94a7e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-05T07:34:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 -- 172.21.15.43:0/4092518860 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8100310 msgr2=0x7f8fe8100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 --2- 172.21.15.43:0/4092518860 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8100310 0x7f8fe8100790 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009990 tx=0x7f8fdc009ca0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 -- 172.21.15.43:0/4092518860 shutdown_connections 2024-02-05T07:34:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 --2- 172.21.15.43:0/4092518860 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100cd0 0x7f8fe810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 --2- 172.21.15.43:0/4092518860 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fe8106d90 0x7f8fe8107170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 --2- 172.21.15.43:0/4092518860 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8100310 0x7f8fe8100790 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 -- 172.21.15.43:0/4092518860 >> 172.21.15.43:0/4092518860 conn(0x7f8fe8077220 msgr2=0x7f8fe80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 -- 172.21.15.43:0/4092518860 shutdown_connections 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.402+0000 7f8fedda9700 1 -- 172.21.15.43:0/4092518860 wait complete. 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 Processor -- start 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 -- start start 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100310 0x7f8fe819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fe8100cd0 0x7f8fe819db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8106d90 0x7f8fe8197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8197ef0 con 0x7f8fe8100310 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8198060 con 0x7f8fe8100cd0 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fedda9700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8198220 con 0x7f8fe8106d90 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fe77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100310 0x7f8fe819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:34.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fe77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100310 0x7f8fe819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41898/0 (socket says 172.21.15.43:41898) 2024-02-05T07:34:34.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fe77fe700 1 -- 172.21.15.43:0/1668871374 learned_addr learned my addr 172.21.15.43:0/1668871374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:34.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fe6ffd700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fe8100cd0 0x7f8fe819db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:34.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.403+0000 7f8fe7fff700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8106d90 0x7f8fe8197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:34.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe77fe700 1 -- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8106d90 msgr2=0x7f8fe8197740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:34.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe77fe700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8106d90 0x7f8fe8197740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe77fe700 1 -- 172.21.15.43:0/1668871374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fe8100cd0 msgr2=0x7f8fe819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:34.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe77fe700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fe8100cd0 0x7f8fe819db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe77fe700 1 -- 172.21.15.43:0/1668871374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0009c40 con 0x7f8fe8100310 2024-02-05T07:34:34.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe6ffd700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fe8100cd0 0x7f8fe819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe77fe700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100310 0x7f8fe819d620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9c0 tx=0x7f8fd800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe4ff9700 1 -- 172.21.15.43:0/1668871374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80097b0 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe4ff9700 1 -- 172.21.15.43:0/1668871374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8fd800ee60 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fe4ff9700 1 -- 172.21.15.43:0/1668871374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80053a0 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fdc0095f0 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.404+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe8101420 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.406+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe810a2f0 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.406+0000 7f8fe4ff9700 1 -- 172.21.15.43:0/1668871374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 17) v1 ==== 91471+0+0 (secure 0 0 0) 0x7f8fd8005580 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.406+0000 7f8fe4ff9700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 0x7f8fd006ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.406+0000 7f8fe6ffd700 1 -- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 msgr2=0x7f8fd006ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/629017799 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.406+0000 7f8fe4ff9700 1 -- 172.21.15.43:0/1668871374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7207+0+0 (secure 0 0 0) 0x7f8fd8014070 con 0x7f8fe8100310 2024-02-05T07:34:34.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.406+0000 7f8fe6ffd700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 0x7f8fd006ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:34:34.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.411+0000 7f8fe4ff9700 1 -- 172.21.15.43:0/1668871374 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fd8056ef0 con 0x7f8fe8100310 2024-02-05T07:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.607+0000 7f8fe6ffd700 1 -- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 msgr2=0x7f8fd006ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/629017799 2024-02-05T07:34:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.607+0000 7f8fe6ffd700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 0x7f8fd006ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:34:34.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.628+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fe80689e0 con 0x7f8fe8100310 2024-02-05T07:34:34.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.629+0000 7f8fe4ff9700 1 -- 172.21.15.43:0/1668871374 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+22695 (secure 0 0 0) 0x7f8fd805a5b0 con 0x7f8fe8100310 2024-02-05T07:34:34.631 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:34:34.631 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"f211dd04-c3f7-11ee-95b6-87774f69a715","created":"2024-02-05T07:28:14.907995+0000","modified":"2024-02-05T07:34:16.944988+0000","last_up_change":"2024-02-05T07:33:42.934794+0000","last_in_change":"2024-02-05T07:33:25.480400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T07:31:37.376556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-05T07:33:55.943256+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-05T07:33:56.489675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-05T07:33:58.313854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-05T07:34:00.266993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-05T07:34:02.335435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c0b3234-6471-45a1-ad20-a2e96fd69264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6803","nonce":2715902880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6805","nonce":2715902880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6809","nonce":2715902880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6807","nonce":2715902880}]},"public_addr":"172.21.15.43:6803/2715902880","cluster_addr":"172.21.15.43:6805/2715902880","heartbeat_back_addr":"172.21.15.43:6809/2715902880","heartbeat_front_addr":"172.21.15.43:6807/2715902880","state":["exists","up"]},{"osd":1,"uuid":"903ff5b0-7d34-4899-8a55-50719f1a09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6811","nonce":193417229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6813","nonce":193417229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6817","nonce":193417229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6815","nonce":193417229}]},"public_addr":"172.21.15.43:6811/193417229","cluster_addr":"172.21.15.43:6813/193417229","heartbeat_back_addr":"172.21.15.43:6817/193417229","heartbeat_front_addr":"172.21.15.43:6815/193417229","state":["exists","up"]},{"osd":2,"uuid":"3527ab76-91de-487e-b5b1-6b5c1933b880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6819","nonce":2847356121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6821","nonce":2847356121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6825","nonce":2847356121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6823","nonce":2847356121}]},"public_addr":"172.21.15.43:6819/2847356121","cluster_addr":"172.21.15.43:6821/2847356121","heartbeat_back_addr":"172.21.15.43:6825/2847356121","heartbeat_front_addr":"172.21.15.43:6823/2847356121","state":["exists","up"]},{"osd":3,"uuid":"2a1431fa-3b8a-4aec-aaf4-58fa8ff28175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6827","nonce":4103809378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6829","nonce":4103809378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6833","nonce":4103809378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6831","nonce":4103809378}]},"public_addr":"172.21.15.43:6827/4103809378","cluster_addr":"172.21.15.43:6829/4103809378","heartbeat_back_addr":"172.21.15.43:6833/4103809378","heartbeat_front_addr":"172.21.15.43:6831/4103809378","state":["exists","up"]},{"osd":4,"uuid":"79c3dcf9-499c-4223-a1ce-b8cace5dc16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6801","nonce":2018737057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6803","nonce":2018737057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6807","nonce":2018737057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6805","nonce":2018737057}]},"public_addr":"172.21.15.50:6801/2018737057","cluster_addr":"172.21.15.50:6803/2018737057","heartbeat_back_addr":"172.21.15.50:6807/2018737057","heartbeat_front_addr":"172.21.15.50:6805/2018737057","state":["exists","up"]},{"osd":5,"uuid":"28fb0eeb-fce2-43e1-b5ee-e298db27ea98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6809","nonce":1047482547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6811","nonce":1047482547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6815","nonce":1047482547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6813","nonce":1047482547}]},"public_addr":"172.21.15.50:6809/1047482547","cluster_addr":"172.21.15.50:6811/1047482547","heartbeat_back_addr":"172.21.15.50:6815/1047482547","heartbeat_front_addr":"172.21.15.50:6813/1047482547","state":["exists","up"]},{"osd":6,"uuid":"b63e0e96-54e3-482b-91f7-4662d8856477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6817","nonce":2200245577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6819","nonce":2200245577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6823","nonce":2200245577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6821","nonce":2200245577}]},"public_addr":"172.21.15.50:6817/2200245577","cluster_addr":"172.21.15.50:6819/2200245577","heartbeat_back_addr":"172.21.15.50:6823/2200245577","heartbeat_front_addr":"172.21.15.50:6821/2200245577","state":["exists","up"]},{"osd":7,"uuid":"91f453b0-c206-41a4-ae3c-422d2f3b8b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6825","nonce":1826641620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6827","nonce":1826641620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6831","nonce":1826641620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6829","nonce":1826641620}]},"public_addr":"172.21.15.50:6825/1826641620","cluster_addr":"172.21.15.50:6827/1826641620","heartbeat_back_addr":"172.21.15.50:6831/1826641620","heartbeat_front_addr":"172.21.15.50:6829/1826641620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:30:41.016171+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:06.930315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:32.938609+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:59.536430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:32:23.858054+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:32:49.859628+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:33:15.590364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:33:41.632379+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3899220204":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6800/4155203878":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6801/4155203878":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6801/1476147457":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/1315802129":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6800/1476147457":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/1334642393":"2024-02-06T07:29:15.031939+0000","172.21.15.43:0/2189512325":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/3337674052":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/2734005924":"2024-02-06T07:28:47.869985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T07:34:34.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 msgr2=0x7f8fd006ff70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 0x7f8fd006ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100310 msgr2=0x7f8fe819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100310 0x7f8fe819d620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800e9c0 tx=0x7f8fd800ecd0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 shutdown_connections 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7f8fd006dab0 0x7f8fd006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f8fe8106d90 0x7f8fe8197740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fe8100310 0x7f8fe819d620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.631+0000 7f8fedda9700 1 --2- 172.21.15.43:0/1668871374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f8fe8100cd0 0x7f8fe819db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.632+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 >> 172.21.15.43:0/1668871374 conn(0x7f8fe8077220 msgr2=0x7f8fe8105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:34.633 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:34.451+0000 7f0e94a7e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-05T07:34:34.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.632+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 shutdown_connections 2024-02-05T07:34:34.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:34.632+0000 7f8fedda9700 1 -- 172.21.15.43:0/1668871374 wait complete. 2024-02-05T07:34:34.682 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:34 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:34.547+0000 7fa406c96200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:34:34.960 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:34.632+0000 7f0e94a7e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-05T07:34:34.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1668871374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:34:34.961 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1668871374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:34:35.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1668871374' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:34:35.280 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-05T07:34:35.280 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-05T07:34:35.528 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:35.822 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:35.539+0000 7fa406c96200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:34:35.822 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:35.672+0000 7fa406c96200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:34:35.922 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:35 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:35.660+0000 7f0e94a7e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-05T07:34:35.922 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:35 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:35.802+0000 7f0e94a7e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-05T07:34:36.244 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:36 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:36.150+0000 7f0e94a7e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:34:36.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:36.012+0000 7fa406c96200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-05T07:34:36.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:36.104+0000 7fa406c96200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:34:36.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:36.304+0000 7fa406c96200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:34:36.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:36 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:36.243+0000 7f0e94a7e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-05T07:34:36.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:36 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:36.449+0000 7f0e94a7e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:34:36.572 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:36.494+0000 7fa406c96200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:34:36.947 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:36 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:36.641+0000 7f0e94a7e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-05T07:34:37.051 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:36.785+0000 7fa406c96200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:34:37.051 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:37 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:37.049+0000 7fa406c96200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:34:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 -- 172.21.15.43:0/870128922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4100eb0 msgr2=0x7fcde4101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/870128922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4100eb0 0x7fcde4101290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcdd4009990 tx=0x7fcdd4009ca0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 -- 172.21.15.43:0/870128922 shutdown_connections 2024-02-05T07:34:37.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/870128922 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 0x7fcde410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/870128922 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4100eb0 0x7fcde4101290 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/870128922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde4101860 0x7fcde4105820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 -- 172.21.15.43:0/870128922 >> 172.21.15.43:0/870128922 conn(0x7fcde4076c70 msgr2=0x7fcde4077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 -- 172.21.15.43:0/870128922 shutdown_connections 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.186+0000 7fcde9c1e700 1 -- 172.21.15.43:0/870128922 wait complete. 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 Processor -- start 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 -- start start 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde4100eb0 0x7fcde419d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4101860 0x7fcde419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 0x7fcde41975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4197c20 con 0x7fcde4100eb0 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4197d60 con 0x7fcde4105f00 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde9c1e700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde4197f20 con 0x7fcde4101860 2024-02-05T07:34:37.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 0x7fcde41975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde3fff700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 0x7fcde41975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.43:51478/0 (socket says 172.21.15.43:51478) 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.187+0000 7fcde3fff700 1 -- 172.21.15.43:0/2560253568 learned_addr learned my addr 172.21.15.43:0/2560253568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde2ffd700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4101860 0x7fcde419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde37fe700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde4100eb0 0x7fcde419d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde3fff700 1 -- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4101860 msgr2=0x7fcde419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde3fff700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4101860 0x7fcde419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde3fff700 1 -- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde4100eb0 msgr2=0x7fcde419d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde3fff700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde4100eb0 0x7fcde419d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde3fff700 1 -- 172.21.15.43:0/2560253568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd8009520 con 0x7fcde4105f00 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde2ffd700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4101860 0x7fcde419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde3fff700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 0x7fcde41975c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00ef30 tx=0x7fcddc00c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde0ff9700 1 -- 172.21.15.43:0/2560253568 <== mon.1 v2:172.21.15.50:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc018070 con 0x7fcde4105f00 2024-02-05T07:34:37.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde0ff9700 1 -- 172.21.15.43:0/2560253568 <== mon.1 v2:172.21.15.50:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcddc00f040 con 0x7fcde4105f00 2024-02-05T07:34:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd40095f0 con 0x7fcde4105f00 2024-02-05T07:34:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde0ff9700 1 -- 172.21.15.43:0/2560253568 <== mon.1 v2:172.21.15.50:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc003c60 con 0x7fcde4105f00 2024-02-05T07:34:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.188+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde41a42d0 con 0x7fcde4105f00 2024-02-05T07:34:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.189+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde40689e0 con 0x7fcde4105f00 2024-02-05T07:34:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.190+0000 7fcde0ff9700 1 -- 172.21.15.43:0/2560253568 <== mon.1 v2:172.21.15.50:3300/0 4 ==== mgrmap(e 17) v1 ==== 91471+0+0 (secure 0 0 0) 0x7fcddc007500 con 0x7fcde4105f00 2024-02-05T07:34:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.190+0000 7fcde0ff9700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 0x7fcdcc06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.190+0000 7fcde37fe700 1 -- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 msgr2=0x7fcdcc06ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/629017799 2024-02-05T07:34:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.190+0000 7fcde0ff9700 1 -- 172.21.15.43:0/2560253568 <== mon.1 v2:172.21.15.50:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7207+0+0 (secure 0 0 0) 0x7fcddc08d520 con 0x7fcde4105f00 2024-02-05T07:34:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.190+0000 7fcde37fe700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 0x7fcdcc06ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-05T07:34:37.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.195+0000 7fcde0ff9700 1 -- 172.21.15.43:0/2560253568 <== mon.1 v2:172.21.15.50:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcddc0568e0 con 0x7fcde4105f00 2024-02-05T07:34:37.221 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:36 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:36.946+0000 7f0e94a7e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-05T07:34:37.322 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:37 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:37.291+0000 7fa406c96200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:34:37.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.391+0000 7fcde37fe700 1 -- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 msgr2=0x7fcdcc06ff70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.43:6800/629017799 2024-02-05T07:34:37.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.391+0000 7fcde37fe700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 0x7fcdcc06ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-05T07:34:37.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.420+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcde4110140 con 0x7fcde4105f00 2024-02-05T07:34:37.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.422+0000 7fcde0ff9700 1 -- 172.21.15.43:0/2560253568 <== mon.1 v2:172.21.15.50:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+22695 (secure 0 0 0) 0x7fcddc059fa0 con 0x7fcde4105f00 2024-02-05T07:34:37.424 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:34:37.424 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"f211dd04-c3f7-11ee-95b6-87774f69a715","created":"2024-02-05T07:28:14.907995+0000","modified":"2024-02-05T07:34:16.944988+0000","last_up_change":"2024-02-05T07:33:42.934794+0000","last_in_change":"2024-02-05T07:33:25.480400+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-05T07:31:37.376556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-02-05T07:33:55.943256+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-05T07:33:56.489675+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-05T07:33:58.313854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-05T07:34:00.266993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-05T07:34:02.335435+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0c0b3234-6471-45a1-ad20-a2e96fd69264","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6803","nonce":2715902880}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6805","nonce":2715902880}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6809","nonce":2715902880}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":2715902880},{"type":"v1","addr":"172.21.15.43:6807","nonce":2715902880}]},"public_addr":"172.21.15.43:6803/2715902880","cluster_addr":"172.21.15.43:6805/2715902880","heartbeat_back_addr":"172.21.15.43:6809/2715902880","heartbeat_front_addr":"172.21.15.43:6807/2715902880","state":["exists","up"]},{"osd":1,"uuid":"903ff5b0-7d34-4899-8a55-50719f1a09da","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6811","nonce":193417229}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6813","nonce":193417229}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6817","nonce":193417229}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":193417229},{"type":"v1","addr":"172.21.15.43:6815","nonce":193417229}]},"public_addr":"172.21.15.43:6811/193417229","cluster_addr":"172.21.15.43:6813/193417229","heartbeat_back_addr":"172.21.15.43:6817/193417229","heartbeat_front_addr":"172.21.15.43:6815/193417229","state":["exists","up"]},{"osd":2,"uuid":"3527ab76-91de-487e-b5b1-6b5c1933b880","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6818","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6819","nonce":2847356121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6821","nonce":2847356121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6825","nonce":2847356121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6822","nonce":2847356121},{"type":"v1","addr":"172.21.15.43:6823","nonce":2847356121}]},"public_addr":"172.21.15.43:6819/2847356121","cluster_addr":"172.21.15.43:6821/2847356121","heartbeat_back_addr":"172.21.15.43:6825/2847356121","heartbeat_front_addr":"172.21.15.43:6823/2847356121","state":["exists","up"]},{"osd":3,"uuid":"2a1431fa-3b8a-4aec-aaf4-58fa8ff28175","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6826","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6827","nonce":4103809378}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6829","nonce":4103809378}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6833","nonce":4103809378}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6830","nonce":4103809378},{"type":"v1","addr":"172.21.15.43:6831","nonce":4103809378}]},"public_addr":"172.21.15.43:6827/4103809378","cluster_addr":"172.21.15.43:6829/4103809378","heartbeat_back_addr":"172.21.15.43:6833/4103809378","heartbeat_front_addr":"172.21.15.43:6831/4103809378","state":["exists","up"]},{"osd":4,"uuid":"79c3dcf9-499c-4223-a1ce-b8cace5dc16a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6801","nonce":2018737057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6803","nonce":2018737057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6807","nonce":2018737057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2018737057},{"type":"v1","addr":"172.21.15.50:6805","nonce":2018737057}]},"public_addr":"172.21.15.50:6801/2018737057","cluster_addr":"172.21.15.50:6803/2018737057","heartbeat_back_addr":"172.21.15.50:6807/2018737057","heartbeat_front_addr":"172.21.15.50:6805/2018737057","state":["exists","up"]},{"osd":5,"uuid":"28fb0eeb-fce2-43e1-b5ee-e298db27ea98","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6809","nonce":1047482547}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6811","nonce":1047482547}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6815","nonce":1047482547}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":1047482547},{"type":"v1","addr":"172.21.15.50:6813","nonce":1047482547}]},"public_addr":"172.21.15.50:6809/1047482547","cluster_addr":"172.21.15.50:6811/1047482547","heartbeat_back_addr":"172.21.15.50:6815/1047482547","heartbeat_front_addr":"172.21.15.50:6813/1047482547","state":["exists","up"]},{"osd":6,"uuid":"b63e0e96-54e3-482b-91f7-4662d8856477","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6817","nonce":2200245577}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6819","nonce":2200245577}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6823","nonce":2200245577}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":2200245577},{"type":"v1","addr":"172.21.15.50:6821","nonce":2200245577}]},"public_addr":"172.21.15.50:6817/2200245577","cluster_addr":"172.21.15.50:6819/2200245577","heartbeat_back_addr":"172.21.15.50:6823/2200245577","heartbeat_front_addr":"172.21.15.50:6821/2200245577","state":["exists","up"]},{"osd":7,"uuid":"91f453b0-c206-41a4-ae3c-422d2f3b8b0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6825","nonce":1826641620}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6827","nonce":1826641620}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6831","nonce":1826641620}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":1826641620},{"type":"v1","addr":"172.21.15.50:6829","nonce":1826641620}]},"public_addr":"172.21.15.50:6825/1826641620","cluster_addr":"172.21.15.50:6827/1826641620","heartbeat_back_addr":"172.21.15.50:6831/1826641620","heartbeat_front_addr":"172.21.15.50:6829/1826641620","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:30:41.016171+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:06.930315+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:32.938609+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:31:59.536430+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:32:23.858054+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:32:49.859628+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:33:15.590364+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-05T07:33:41.632379+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.7","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.14","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/3899220204":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6800/4155203878":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6801/4155203878":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6801/1476147457":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/1315802129":"2024-02-06T07:29:15.031939+0000","172.21.15.43:6800/1476147457":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/1334642393":"2024-02-06T07:29:15.031939+0000","172.21.15.43:0/2189512325":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/3337674052":"2024-02-06T07:28:47.869985+0000","172.21.15.43:0/2734005924":"2024-02-06T07:28:47.869985+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.424+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 msgr2=0x7fcdcc06ff70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.424+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 0x7fcdcc06ff70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.424+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 msgr2=0x7fcde41975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.424+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 0x7fcde41975c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fcddc00ef30 tx=0x7fcddc00c450 comp rx=0 tx=0).stop 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.424+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 shutdown_connections 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.424+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:6800/629017799,v1:172.21.15.43:6801/629017799] conn(0x7fcdcc06dab0 0x7fcdcc06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.425+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fcde4105f00 0x7fcde41975c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.425+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcde4100eb0 0x7fcde419d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.425+0000 7fcde9c1e700 1 --2- 172.21.15.43:0/2560253568 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fcde4101860 0x7fcde419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.425+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 >> 172.21.15.43:0/2560253568 conn(0x7fcde4076c70 msgr2=0x7fcde406a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.425+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 shutdown_connections 2024-02-05T07:34:37.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:37.425+0000 7fcde9c1e700 1 -- 172.21.15.43:0/2560253568 wait complete. 2024-02-05T07:34:37.475 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:37 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:37.237+0000 7f0e94a7e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-05T07:34:37.775 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:37 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:37.537+0000 7f0e94a7e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-05T07:34:37.775 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2560253568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:34:37.776 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2560253568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:34:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2560253568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-05T07:34:38.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-05T07:34:38.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-05T07:34:38.189 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-05T07:34:38.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-05T07:34:38.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-05T07:34:38.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-05T07:34:38.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-05T07:34:38.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-05T07:34:38.404 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:38 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:38.118+0000 7fa406c96200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:34:38.750 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:38.783 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:38.790 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:38.792 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:38.880 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:38 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:38.571+0000 7f0e94a7e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-05T07:34:38.995 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:38 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:38.706+0000 7fa406c96200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:34:38.995 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:38 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:38.807+0000 7fa406c96200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:34:38.999 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:39.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[41444]: Standby manager daemon x restarted 2024-02-05T07:34:39.257 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[41444]: Standby manager daemon x started 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[41444]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[29879]: Standby manager daemon x restarted 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[29879]: Standby manager daemon x started 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:34:39.258 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-mon[29879]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:34:39.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-mon[33492]: Standby manager daemon x restarted 2024-02-05T07:34:39.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-mon[33492]: Standby manager daemon x started 2024-02-05T07:34:39.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-mon[33492]: from='mgr.? 172.21.15.50:0/4175321783' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:39.033+0000 7fa406c96200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: [05/Feb/2024:07:34:39] ENGINE Bus STARTING 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: CherryPy Checker: 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: The Application mounted at '' has an empty config. 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: [05/Feb/2024:07:34:39] ENGINE Serving on http://:::9283 2024-02-05T07:34:39.323 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 07:34:39 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x[35600]: [05/Feb/2024:07:34:39] ENGINE Bus STARTED 2024-02-05T07:34:39.560 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:39.256+0000 7f0e94a7e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-05T07:34:39.560 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:39.363+0000 7f0e94a7e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-05T07:34:39.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:39.598+0000 7f0e94a7e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-05T07:34:39.813 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:34:39] ENGINE Bus STARTING 2024-02-05T07:34:40.110 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: CherryPy Checker: 2024-02-05T07:34:40.110 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: The Application mounted at '' has an empty config. 2024-02-05T07:34:40.110 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T07:34:40.110 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:34:39] ENGINE Serving on http://:::9283 2024-02-05T07:34:40.110 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:34:39] ENGINE Bus STARTED 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: Active manager daemon y restarted 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: Activating manager daemon y 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: osdmap e74: 8 total, 8 up, 8 in 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: mgrmap e19: y(active, starting, since 0.00763389s), standbys: x 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:34:40.467 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: Manager daemon y is now available 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:40.468 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: Active manager daemon y restarted 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: Activating manager daemon y 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: osdmap e74: 8 total, 8 up, 8 in 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: mgrmap e19: y(active, starting, since 0.00763389s), standbys: x 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:34:40.469 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:34:40.470 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: Manager daemon y is now available 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:34:40.471 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: mgrmap e18: y(active, since 5m), standbys: x 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: Active manager daemon y restarted 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: Activating manager daemon y 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: osdmap e74: 8 total, 8 up, 8 in 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: mgrmap e19: y(active, starting, since 0.00763389s), standbys: x 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-05T07:34:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: Manager daemon y is now available 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:40.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:34:40.575 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:34:40.575 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-05T07:34:40.575 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:34:40.575 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-05T07:34:40.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:34:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[41444]: [05/Feb/2024:07:34:40] ENGINE Bus STARTING 2024-02-05T07:34:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[41444]: [05/Feb/2024:07:34:40] ENGINE Serving on https://172.21.15.43:7150 2024-02-05T07:34:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[41444]: [05/Feb/2024:07:34:40] ENGINE Serving on http://172.21.15.43:8765 2024-02-05T07:34:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[41444]: [05/Feb/2024:07:34:40] ENGINE Bus STARTED 2024-02-05T07:34:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[41444]: mgrmap e20: y(active, since 1.0103s), standbys: x 2024-02-05T07:34:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[41444]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:41.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[29879]: [05/Feb/2024:07:34:40] ENGINE Bus STARTING 2024-02-05T07:34:41.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[29879]: [05/Feb/2024:07:34:40] ENGINE Serving on https://172.21.15.43:7150 2024-02-05T07:34:41.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[29879]: [05/Feb/2024:07:34:40] ENGINE Serving on http://172.21.15.43:8765 2024-02-05T07:34:41.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[29879]: [05/Feb/2024:07:34:40] ENGINE Bus STARTED 2024-02-05T07:34:41.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[29879]: mgrmap e20: y(active, since 1.0103s), standbys: x 2024-02-05T07:34:41.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:41.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:41 smithi043 ceph-mon[29879]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:41.938 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:41 smithi050 ceph-mon[33492]: [05/Feb/2024:07:34:40] ENGINE Bus STARTING 2024-02-05T07:34:41.938 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:41 smithi050 ceph-mon[33492]: [05/Feb/2024:07:34:40] ENGINE Serving on https://172.21.15.43:7150 2024-02-05T07:34:41.938 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:41 smithi050 ceph-mon[33492]: [05/Feb/2024:07:34:40] ENGINE Serving on http://172.21.15.43:8765 2024-02-05T07:34:41.938 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:41 smithi050 ceph-mon[33492]: [05/Feb/2024:07:34:40] ENGINE Bus STARTED 2024-02-05T07:34:41.938 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:41 smithi050 ceph-mon[33492]: mgrmap e20: y(active, since 1.0103s), standbys: x 2024-02-05T07:34:41.938 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:41 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:41.939 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:41 smithi050 ceph-mon[33492]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:43.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:42 smithi043 ceph-mon[29879]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:43.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:42 smithi043 ceph-mon[41444]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:42 smithi050 ceph-mon[33492]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:43.779 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:43 smithi043 ceph-mon[29879]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-05T07:34:43.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:43.780 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:43.780 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:43 smithi043 ceph-mon[41444]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-05T07:34:43.780 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:43.780 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:44.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:43 smithi050 ceph-mon[33492]: mgrmap e21: y(active, since 3s), standbys: x 2024-02-05T07:34:44.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:44.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.224 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.225 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[29879]: mgrmap e22: y(active, since 5s), standbys: x 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:45 smithi043 ceph-mon[41444]: mgrmap e22: y(active, since 5s), standbys: x 2024-02-05T07:34:45.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:34:45.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:45 smithi050 ceph-mon[33492]: mgrmap e22: y(active, since 5s), standbys: x 2024-02-05T07:34:45.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/89646848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c8106c60 msgr2=0x7ff0c8107040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 --2- 172.21.15.43:0/89646848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c8106c60 0x7ff0c8107040 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0009990 tx=0x7ff0b0009ca0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/89646848 shutdown_connections 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 --2- 172.21.15.43:0/89646848 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0c81015c0 0x7ff0c810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 --2- 172.21.15.43:0/89646848 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c8106c60 0x7ff0c8107040 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 --2- 172.21.15.43:0/89646848 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c8100c20 0x7ff0c8101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/89646848 >> 172.21.15.43:0/89646848 conn(0x7ff0c8077b50 msgr2=0x7ff0c8077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/89646848 shutdown_connections 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.653+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/89646848 wait complete. 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 Processor -- start 2024-02-05T07:34:45.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 -- start start 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0c8100c20 0x7ff0c81986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c81015c0 0x7ff0c8198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c8106c60 0x7ff0c81a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c81994a0 con 0x7ff0c81015c0 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c81a3cb0 con 0x7ff0c8106c60 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0ce8cc700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c81a3df0 con 0x7ff0c8100c20 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c81015c0 0x7ff0c8198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0c77fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c81015c0 0x7ff0c8198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49224/0 (socket says 172.21.15.43:49224) 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0c77fe700 1 -- 172.21.15.43:0/2003982866 learned_addr learned my addr 172.21.15.43:0/2003982866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0c7fff700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0c8100c20 0x7ff0c81986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0cce69700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c8106c60 0x7ff0c81a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.654+0000 7ff0c77fe700 1 -- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0c8100c20 msgr2=0x7ff0c81986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c77fe700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff0c8100c20 0x7ff0c81986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c77fe700 1 -- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c8106c60 msgr2=0x7ff0c81a3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c77fe700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c8106c60 0x7ff0c81a3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c77fe700 1 -- 172.21.15.43:0/2003982866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b00095f0 con 0x7ff0c81015c0 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0cce69700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff0c8106c60 0x7ff0c81a3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c77fe700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c81015c0 0x7ff0c8198c30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800b530 tx=0x7ff0b800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8011830 con 0x7ff0c81015c0 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c81a4080 con 0x7ff0c81015c0 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff0b8011e10 con 0x7ff0c81015c0 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b8010e30 con 0x7ff0c81015c0 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.655+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c81a45d0 con 0x7ff0c81015c0 2024-02-05T07:34:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.656+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0c8110480 con 0x7ff0c81015c0 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.656+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7ff0b8011990 con 0x7ff0c81015c0 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0c57fa700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff0b406dac0 0x7ff0b406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7ff0b808d4c0 con 0x7ff0c81015c0 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0c7fff700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff0b406dac0 0x7ff0b406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0c57fa700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] conn(0x7ff0b4077950 0x7ff0b4079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 --> [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff0b407a4c0 con 0x7ff0b4077950 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0b808fb80 con 0x7ff0c81015c0 2024-02-05T07:34:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0c7fff700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff0b406dac0 0x7ff0b406ff80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000b2e0 tx=0x7ff0b0005040 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.657+0000 7ff0cce69700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] conn(0x7ff0b4077950 0x7ff0b4079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.658+0000 7ff0cce69700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] conn(0x7ff0b4077950 0x7ff0b4079df0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.659+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== osd.6 v2:172.21.15.50:6816/2200245577 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff0b407a4c0 con 0x7ff0b4077950 2024-02-05T07:34:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.675+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 --> [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0c80623b0 con 0x7ff0b4077950 2024-02-05T07:34:45.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0c57fa700 1 -- 172.21.15.43:0/2003982866 <== osd.6 v2:172.21.15.50:6816/2200245577 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff0c80623b0 con 0x7ff0b4077950 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] conn(0x7ff0b4077950 msgr2=0x7ff0b4079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.50:6816/2200245577,v1:172.21.15.50:6817/2200245577] conn(0x7ff0b4077950 0x7ff0b4079df0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff0b406dac0 msgr2=0x7ff0b406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff0b406dac0 0x7ff0b406ff80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff0b000b2e0 tx=0x7ff0b0005040 comp rx=0 tx=0).stop 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c81015c0 msgr2=0x7ff0c8198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 --2- 172.21.15.43:0/2003982866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff0c81015c0 0x7ff0c8198c30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff0b800b530 tx=0x7ff0b800b840 comp rx=0 tx=0).stop 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0cce69700 1 -- 172.21.15.43:0/2003982866 reap_dead start 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 shutdown_connections 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.676+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 >> 172.21.15.43:0/2003982866 conn(0x7ff0c8077b50 msgr2=0x7ff0c80ffb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.677+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 shutdown_connections 2024-02-05T07:34:45.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.677+0000 7ff0ce8cc700 1 -- 172.21.15.43:0/2003982866 wait complete. 2024-02-05T07:34:45.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 -- 172.21.15.43:0/955077489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006ae70 msgr2=0x7fd300106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 --2- 172.21.15.43:0/955077489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006ae70 0x7fd300106970 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40098f0 tx=0x7fd2f4009c00 comp rx=0 tx=0).stop 2024-02-05T07:34:45.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 -- 172.21.15.43:0/955077489 shutdown_connections 2024-02-05T07:34:45.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 --2- 172.21.15.43:0/955077489 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd3001070a0 0x7fd30010ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 --2- 172.21.15.43:0/955077489 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd30006a4c0 0x7fd30006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 --2- 172.21.15.43:0/955077489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006ae70 0x7fd300106970 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 -- 172.21.15.43:0/955077489 >> 172.21.15.43:0/955077489 conn(0x7fd300076c70 msgr2=0x7fd300077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 -- 172.21.15.43:0/955077489 shutdown_connections 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 -- 172.21.15.43:0/955077489 wait complete. 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.859+0000 7fd30618b700 1 Processor -- start 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd30618b700 1 -- start start 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd30618b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006a4c0 0x7fd30019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd30618b700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd30006ae70 0x7fd30019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd30618b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3001070a0 0x7fd300197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd30618b700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300197e40 con 0x7fd30006a4c0 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd30618b700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd300197fb0 con 0x7fd3001070a0 2024-02-05T07:34:45.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd30618b700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd300198170 con 0x7fd30006ae70 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006a4c0 0x7fd30019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2feffd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd30006ae70 0x7fd30019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006a4c0 0x7fd30019d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49250/0 (socket says 172.21.15.43:49250) 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 -- 172.21.15.43:0/2713446455 learned_addr learned my addr 172.21.15.43:0/2713446455 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 -- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd30006ae70 msgr2=0x7fd30019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd30006ae70 0x7fd30019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 -- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3001070a0 msgr2=0x7fd300197690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2fffff700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3001070a0 0x7fd300197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd3001070a0 0x7fd300197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 -- 172.21.15.43:0/2713446455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8009d00 con 0x7fd30006a4c0 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.860+0000 7fd2ff7fe700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006a4c0 0x7fd30019d570 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000d980 tx=0x7fd2f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.861+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0004220 con 0x7fd30006a4c0 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.861+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd2f0007640 con 0x7fd30006a4c0 2024-02-05T07:34:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.861+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f4009520 con 0x7fd30006a4c0 2024-02-05T07:34:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.861+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00054a0 con 0x7fd30006a4c0 2024-02-05T07:34:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.861+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3001a44f0 con 0x7fd30006a4c0 2024-02-05T07:34:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.862+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd3001115a0 con 0x7fd30006a4c0 2024-02-05T07:34:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.862+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fd2f0005600 con 0x7fd30006a4c0 2024-02-05T07:34:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.862+0000 7fd2fcff9700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd2e806dac0 0x7fd2e806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fd2f008d690 con 0x7fd30006a4c0 2024-02-05T07:34:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2feffd700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd2e806dac0 0x7fd2e806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2fcff9700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] conn(0x7fd2e8077950 0x7fd2e8079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 --> [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd2e807a4c0 con 0x7fd2e8077950 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2f008fd50 con 0x7fd30006a4c0 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2fffff700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] conn(0x7fd2e8077950 0x7fd2e8079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2feffd700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd2e806dac0 0x7fd2e806ff80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40098c0 tx=0x7fd2f4019040 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.863+0000 7fd2fffff700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] conn(0x7fd2e8077950 0x7fd2e8079df0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.864+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== osd.5 v2:172.21.15.50:6808/1047482547 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd2e807a4c0 con 0x7fd2e8077950 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.880+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 --> [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd3000623b0 con 0x7fd2e8077950 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd2fcff9700 1 -- 172.21.15.43:0/2713446455 <== osd.5 v2:172.21.15.50:6808/1047482547 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd3000623b0 con 0x7fd2e8077950 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] conn(0x7fd2e8077950 msgr2=0x7fd2e8079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd30618b700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.50:6808/1047482547,v1:172.21.15.50:6809/1047482547] conn(0x7fd2e8077950 0x7fd2e8079df0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd2e806dac0 msgr2=0x7fd2e806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd30618b700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd2e806dac0 0x7fd2e806ff80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40098c0 tx=0x7fd2f4019040 comp rx=0 tx=0).stop 2024-02-05T07:34:45.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006a4c0 msgr2=0x7fd30019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd30618b700 1 --2- 172.21.15.43:0/2713446455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd30006a4c0 0x7fd30019d570 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000d980 tx=0x7fd2f000dc90 comp rx=0 tx=0).stop 2024-02-05T07:34:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.881+0000 7fd2fffff700 1 -- 172.21.15.43:0/2713446455 reap_dead start 2024-02-05T07:34:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.882+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 shutdown_connections 2024-02-05T07:34:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.882+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 >> 172.21.15.43:0/2713446455 conn(0x7fd300076c70 msgr2=0x7fd3000ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.882+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 shutdown_connections 2024-02-05T07:34:45.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:45.882+0000 7fd30618b700 1 -- 172.21.15.43:0/2713446455 wait complete. 2024-02-05T07:34:46.117 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:46 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi050 to 3761M 2024-02-05T07:34:46.117 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:46 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi050 to 3761M 2024-02-05T07:34:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:46 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi050 to 3761M 2024-02-05T07:34:46.630 INFO:teuthology.orchestra.run.smithi043.stdout:206158430227 2024-02-05T07:34:46.630 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-05T07:34:47.079 INFO:teuthology.orchestra.run.smithi043.stdout:171798691865 2024-02-05T07:34:47.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-05T07:34:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:47 smithi043 ceph-mon[29879]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:47 smithi043 ceph-mon[41444]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:47 smithi050 ceph-mon[33492]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.648+0000 7f51dfa68700 1 -- 172.21.15.43:0/1926960960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 msgr2=0x7f51d810ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.648+0000 7f51dfa68700 1 --2- 172.21.15.43:0/1926960960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 0x7f51d810ebf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009990 tx=0x7f51d0009ca0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.648+0000 7f51dfa68700 1 -- 172.21.15.43:0/1926960960 shutdown_connections 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.648+0000 7f51dfa68700 1 --2- 172.21.15.43:0/1926960960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 0x7f51d810ebf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.648+0000 7f51dfa68700 1 --2- 172.21.15.43:0/1926960960 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51d8107df0 0x7f51d81081d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.649+0000 7f51dfa68700 1 --2- 172.21.15.43:0/1926960960 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51d8101da0 0x7f51d8102220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.649+0000 7f51dfa68700 1 -- 172.21.15.43:0/1926960960 >> 172.21.15.43:0/1926960960 conn(0x7f51d8078e70 msgr2=0x7f51d8079280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.649+0000 7f51dfa68700 1 -- 172.21.15.43:0/1926960960 shutdown_connections 2024-02-05T07:34:47.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.649+0000 7f51dfa68700 1 -- 172.21.15.43:0/1926960960 wait complete. 2024-02-05T07:34:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.649+0000 7f51dfa68700 1 Processor -- start 2024-02-05T07:34:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.649+0000 7f51dfa68700 1 -- start start 2024-02-05T07:34:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dfa68700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51d8101da0 0x7f51d8197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dfa68700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 0x7f51d8197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dfa68700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51d8107df0 0x7f51d819bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:47.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dfa68700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d819c590 con 0x7f51d8102760 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dfa68700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d819c700 con 0x7f51d8101da0 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dfa68700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d819c8c0 con 0x7f51d8107df0 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd003700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 0x7f51d8197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd804700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51d8101da0 0x7f51d8197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51de005700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51d8107df0 0x7f51d819bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd003700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 0x7f51d8197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49254/0 (socket says 172.21.15.43:49254) 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd804700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51d8101da0 0x7f51d8197700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.50:3300/0 says I am v2:172.21.15.43:60772/0 (socket says 172.21.15.43:60772) 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51de005700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51d8107df0 0x7f51d819bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:60600/0 (socket says 172.21.15.43:60600) 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd003700 1 -- 172.21.15.43:0/1083230701 learned_addr learned my addr 172.21.15.43:0/1083230701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd003700 1 -- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51d8107df0 msgr2=0x7f51d819bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd003700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51d8107df0 0x7f51d819bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.650+0000 7f51dd003700 1 -- 172.21.15.43:0/1083230701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51d8101da0 msgr2=0x7f51d8197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51dd003700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51d8101da0 0x7f51d8197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51dd003700 1 -- 172.21.15.43:0/1083230701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d00095f0 con 0x7f51d8102760 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51de005700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f51d8107df0 0x7f51d819bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51dd804700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f51d8101da0 0x7f51d8197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51dd003700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 0x7f51d8197c40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c9b0 tx=0x7f51c800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8007b00 con 0x7f51d8102760 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d819cba0 con 0x7f51d8102760 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f51c800ce50 con 0x7f51d8102760 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8018400 con 0x7f51d8102760 2024-02-05T07:34:47.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.651+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d81a4400 con 0x7f51d8102760 2024-02-05T07:34:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.652+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f51d8111620 con 0x7f51d8102760 2024-02-05T07:34:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f51c800f450 con 0x7f51d8102760 2024-02-05T07:34:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51ceffd700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f51c406dac0 0x7f51c406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f51c808d7b0 con 0x7f51d8102760 2024-02-05T07:34:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51dd804700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f51c406dac0 0x7f51c406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51ceffd700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] conn(0x7f51c40779d0 0x7f51c4079e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 --> [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f51c407a540 con 0x7f51c40779d0 2024-02-05T07:34:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c808fe70 con 0x7f51d8102760 2024-02-05T07:34:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.653+0000 7f51de005700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] conn(0x7f51c40779d0 0x7f51c4079e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.654+0000 7f51dd804700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f51c406dac0 0x7f51c406ff80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009480 tx=0x7f51d400b040 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.654+0000 7f51de005700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] conn(0x7f51c40779d0 0x7f51c4079e70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:47.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.655+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== osd.1 v2:172.21.15.43:6810/193417229 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f51c407a540 con 0x7f51c40779d0 2024-02-05T07:34:47.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.678+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 --> [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f51d80623b0 con 0x7f51c40779d0 2024-02-05T07:34:47.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.679+0000 7f51ceffd700 1 -- 172.21.15.43:0/1083230701 <== osd.1 v2:172.21.15.43:6810/193417229 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f51d80623b0 con 0x7f51c40779d0 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.679+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] conn(0x7f51c40779d0 msgr2=0x7f51c4079e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.679+0000 7f51dfa68700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6810/193417229,v1:172.21.15.43:6811/193417229] conn(0x7f51c40779d0 0x7f51c4079e70 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.679+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f51c406dac0 msgr2=0x7f51c406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.679+0000 7f51dfa68700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f51c406dac0 0x7f51c406ff80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009480 tx=0x7f51d400b040 comp rx=0 tx=0).stop 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.679+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 msgr2=0x7f51d8197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.679+0000 7f51dfa68700 1 --2- 172.21.15.43:0/1083230701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f51d8102760 0x7f51d8197c40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f51c800c9b0 tx=0x7f51c800ccc0 comp rx=0 tx=0).stop 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.680+0000 7f51de005700 1 -- 172.21.15.43:0/1083230701 reap_dead start 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.680+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 shutdown_connections 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.680+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 >> 172.21.15.43:0/1083230701 conn(0x7f51d8078e70 msgr2=0x7f51d8100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.680+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 shutdown_connections 2024-02-05T07:34:47.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:47.680+0000 7f51dfa68700 1 -- 172.21.15.43:0/1083230701 wait complete. 2024-02-05T07:34:47.906 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:48.482 INFO:teuthology.orchestra.run.smithi043.stdout:55834574894 2024-02-05T07:34:48.483 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-05T07:34:49.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:49 smithi043 ceph-mon[29879]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-05T07:34:49.212 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:49 smithi043 ceph-mon[41444]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-05T07:34:49.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:49 smithi050 ceph-mon[33492]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-02-05T07:34:49.506 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:34:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:34:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 -- 172.21.15.43:0/1152221504 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c102000 msgr2=0x7f753c1023e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 --2- 172.21.15.43:0/1152221504 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c102000 0x7f753c1023e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f752c0098f0 tx=0x7f752c009c00 comp rx=0 tx=0).stop 2024-02-05T07:34:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 -- 172.21.15.43:0/1152221504 shutdown_connections 2024-02-05T07:34:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 --2- 172.21.15.43:0/1152221504 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f753c107030 0x7f753c10ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 --2- 172.21.15.43:0/1152221504 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c102000 0x7f753c1023e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:49.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 --2- 172.21.15.43:0/1152221504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f753c1029b0 0x7f753c106950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 -- 172.21.15.43:0/1152221504 >> 172.21.15.43:0/1152221504 conn(0x7f753c078db0 msgr2=0x7f753c0791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 -- 172.21.15.43:0/1152221504 shutdown_connections 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.683+0000 7f7542c43700 1 -- 172.21.15.43:0/1152221504 wait complete. 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 Processor -- start 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 -- start start 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f753c102000 0x7f753c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f753c1029b0 0x7f753c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c107030 0x7f753c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c197e00 con 0x7f753c1029b0 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c197f70 con 0x7f753c102000 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f7542c43700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c198130 con 0x7f753c107030 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f753bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f753c1029b0 0x7f753c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:49.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f75411e0700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c107030 0x7f753c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f753bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f753c1029b0 0x7f753c19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49274/0 (socket says 172.21.15.43:49274) 2024-02-05T07:34:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f753bfff700 1 -- 172.21.15.43:0/2033874420 learned_addr learned my addr 172.21.15.43:0/2033874420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f75409df700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f753c102000 0x7f753c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f75411e0700 1 -- 172.21.15.43:0/2033874420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f753c102000 msgr2=0x7f753c19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f75411e0700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f753c102000 0x7f753c19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:49.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f75411e0700 1 -- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f753c1029b0 msgr2=0x7f753c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f75411e0700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f753c1029b0 0x7f753c19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f75411e0700 1 -- 172.21.15.43:0/2033874420 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75300095f0 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.684+0000 7f753bfff700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f753c1029b0 0x7f753c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.685+0000 7f75411e0700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c107030 0x7f753c197650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7534007f10 tx=0x7f753400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.685+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534004db0 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.685+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f753400f040 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.685+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f752c009520 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.685+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75340058e0 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.685+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c1a4500 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.686+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f753c111490 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.686+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f753400db20 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f7539ffb700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f752406dac0 0x7f752406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f753408d1b0 con 0x7f753c107030 2024-02-05T07:34:49.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f75409df700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f752406dac0 0x7f752406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f7539ffb700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] conn(0x7f75240779d0 0x7f7524079e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 --> [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f752407a540 con 0x7f75240779d0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f753408f710 con 0x7f753c107030 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f753bfff700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] conn(0x7f75240779d0 0x7f7524079e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.687+0000 7f75409df700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f752406dac0 0x7f752406ff80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f752c009fd0 tx=0x7f752c019040 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.688+0000 7f753bfff700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] conn(0x7f75240779d0 0x7f7524079e70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.688+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== osd.2 v2:172.21.15.43:6818/2847356121 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f752407a540 con 0x7f75240779d0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.704+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 --> [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f753c100570 con 0x7f75240779d0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7539ffb700 1 -- 172.21.15.43:0/2033874420 <== osd.2 v2:172.21.15.43:6818/2847356121 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f753c100570 con 0x7f75240779d0 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] conn(0x7f75240779d0 msgr2=0x7f7524079e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6818/2847356121,v1:172.21.15.43:6819/2847356121] conn(0x7f75240779d0 0x7f7524079e70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f752406dac0 msgr2=0x7f752406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f752406dac0 0x7f752406ff80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f752c009fd0 tx=0x7f752c019040 comp rx=0 tx=0).stop 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c107030 msgr2=0x7f753c197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:49.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 --2- 172.21.15.43:0/2033874420 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f753c107030 0x7f753c197650 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7534007f10 tx=0x7f753400d3b0 comp rx=0 tx=0).stop 2024-02-05T07:34:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f75411e0700 1 -- 172.21.15.43:0/2033874420 reap_dead start 2024-02-05T07:34:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 shutdown_connections 2024-02-05T07:34:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 >> 172.21.15.43:0/2033874420 conn(0x7f753c078db0 msgr2=0x7f753c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 shutdown_connections 2024-02-05T07:34:49.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:48.705+0000 7f7542c43700 1 -- 172.21.15.43:0/2033874420 wait complete. 2024-02-05T07:34:50.286 INFO:teuthology.orchestra.run.smithi043.stdout:77309411369 2024-02-05T07:34:50.286 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-05T07:34:50.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 -- 172.21.15.43:0/1079900752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1021f0 msgr2=0x7f499c10db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:50.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 --2- 172.21.15.43:0/1079900752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1021f0 0x7f499c10db50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4994009f90 tx=0x7f499400b330 comp rx=0 tx=0).stop 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 -- 172.21.15.43:0/1079900752 shutdown_connections 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 --2- 172.21.15.43:0/1079900752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1021f0 0x7f499c10db50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 --2- 172.21.15.43:0/1079900752 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f499c105cb0 0x7f499c106090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 --2- 172.21.15.43:0/1079900752 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f499c101830 0x7f499c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 -- 172.21.15.43:0/1079900752 >> 172.21.15.43:0/1079900752 conn(0x7f499c075f70 msgr2=0x7f499c076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.468+0000 7f49a2427700 1 -- 172.21.15.43:0/1079900752 shutdown_connections 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 -- 172.21.15.43:0/1079900752 wait complete. 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 Processor -- start 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 -- start start 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f499c101830 0x7f499c19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1021f0 0x7f499c19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f499c105cb0 0x7f499c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c197ec0 con 0x7f499c1021f0 2024-02-05T07:34:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499c198030 con 0x7f499c105cb0 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f49a2427700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f499c1981f0 con 0x7f499c101830 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.469+0000 7f499bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f499c101830 0x7f499c19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f4993fff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1021f0 0x7f499c19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f499c101830 0x7f499c19d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:60650/0 (socket says 172.21.15.43:60650) 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 -- 172.21.15.43:0/1832170285 learned_addr learned my addr 172.21.15.43:0/1832170285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f49a09c4700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f499c105cb0 0x7f499c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 -- 172.21.15.43:0/1832170285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f499c105cb0 msgr2=0x7f499c197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f499c105cb0 0x7f499c197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 -- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1021f0 msgr2=0x7f499c19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f499c1021f0 0x7f499c19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 -- 172.21.15.43:0/1832170285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994009c40 con 0x7f499c101830 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f499bfff700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f499c101830 0x7f499c19d640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f498c00d950 tx=0x7f498c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c009930 con 0x7f499c101830 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499c1984d0 con 0x7f499c101830 2024-02-05T07:34:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f498c010460 con 0x7f499c101830 2024-02-05T07:34:50.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c00f620 con 0x7f499c101830 2024-02-05T07:34:50.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.470+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499c1a4610 con 0x7f499c101830 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.471+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f499c1104f0 con 0x7f499c101830 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.472+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f498c009a90 con 0x7f499c101830 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.472+0000 7f4999ffb700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f498806dac0 0x7f498806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.472+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f498c08d670 con 0x7f499c101830 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.472+0000 7f4993fff700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f498806dac0 0x7f498806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.472+0000 7f4999ffb700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] conn(0x7f4988077950 0x7f4988079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.472+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 --> [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f498807a4c0 con 0x7f4988077950 2024-02-05T07:34:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.472+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f498c08fd30 con 0x7f499c101830 2024-02-05T07:34:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.473+0000 7f49a09c4700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] conn(0x7f4988077950 0x7f4988079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.473+0000 7f4993fff700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f498806dac0 0x7f498806ff80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f499c198ed0 tx=0x7f4984009310 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.473+0000 7f49a09c4700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] conn(0x7f4988077950 0x7f4988079df0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.474+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== osd.3 v2:172.21.15.43:6826/4103809378 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f498807a4c0 con 0x7f4988077950 2024-02-05T07:34:50.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.491+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 --> [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f499c0623b0 con 0x7f4988077950 2024-02-05T07:34:50.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.491+0000 7f4999ffb700 1 -- 172.21.15.43:0/1832170285 <== osd.3 v2:172.21.15.43:6826/4103809378 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f499c0623b0 con 0x7f4988077950 2024-02-05T07:34:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] conn(0x7f4988077950 msgr2=0x7f4988079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6826/4103809378,v1:172.21.15.43:6827/4103809378] conn(0x7f4988077950 0x7f4988079df0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f498806dac0 msgr2=0x7f498806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f498806dac0 0x7f498806ff80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f499c198ed0 tx=0x7f4984009310 comp rx=0 tx=0).stop 2024-02-05T07:34:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f499c101830 msgr2=0x7f499c19d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:50.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 --2- 172.21.15.43:0/1832170285 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f499c101830 0x7f499c19d640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f498c00d950 tx=0x7f498c00dc60 comp rx=0 tx=0).stop 2024-02-05T07:34:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a09c4700 1 -- 172.21.15.43:0/1832170285 reap_dead start 2024-02-05T07:34:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 shutdown_connections 2024-02-05T07:34:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 >> 172.21.15.43:0/1832170285 conn(0x7f499c075f70 msgr2=0x7f499c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 shutdown_connections 2024-02-05T07:34:50.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:50.492+0000 7f49a2427700 1 -- 172.21.15.43:0/1832170285 wait complete. 2024-02-05T07:34:51.054 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:34:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:34:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:51 smithi043 ceph-mon[29879]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-05T07:34:51.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:51 smithi043 ceph-mon[41444]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-05T07:34:51.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:51 smithi050 ceph-mon[33492]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 13 op/s 2024-02-05T07:34:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 -- 172.21.15.43:0/1476960920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 msgr2=0x7f4bbc10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 --2- 172.21.15.43:0/1476960920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 0x7f4bbc10daa0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4009f90 tx=0x7f4bb400b330 comp rx=0 tx=0).stop 2024-02-05T07:34:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 -- 172.21.15.43:0/1476960920 shutdown_connections 2024-02-05T07:34:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 --2- 172.21.15.43:0/1476960920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 0x7f4bbc10daa0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 --2- 172.21.15.43:0/1476960920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4bbc106c60 0x7f4bbc107040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:51.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 --2- 172.21.15.43:0/1476960920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bbc100c20 0x7f4bbc101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:51.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 -- 172.21.15.43:0/1476960920 >> 172.21.15.43:0/1476960920 conn(0x7f4bbc077b50 msgr2=0x7f4bbc077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:51.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 -- 172.21.15.43:0/1476960920 shutdown_connections 2024-02-05T07:34:51.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 -- 172.21.15.43:0/1476960920 wait complete. 2024-02-05T07:34:51.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 Processor -- start 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.633+0000 7f4bc3851700 1 -- start start 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc3851700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bbc100c20 0x7f4bbc19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc3851700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 0x7f4bbc19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc3851700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4bbc106c60 0x7f4bbc197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc3851700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc197e00 con 0x7f4bbc1015c0 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc3851700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc197f70 con 0x7f4bbc100c20 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc3851700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc198130 con 0x7f4bbc106c60 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4bbc106c60 0x7f4bbc197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc0dec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 0x7f4bbc19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc0dec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 0x7f4bbc19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54036/0 (socket says 172.21.15.43:54036) 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc0dec700 1 -- 172.21.15.43:0/2775495406 learned_addr learned my addr 172.21.15.43:0/2775495406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4bbc106c60 0x7f4bbc197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:58520/0 (socket says 172.21.15.43:58520) 2024-02-05T07:34:51.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc15ed700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bbc100c20 0x7f4bbc19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 -- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bbc100c20 msgr2=0x7f4bbc19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bbc100c20 0x7f4bbc19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 -- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 msgr2=0x7f4bbc19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 0x7f4bbc19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 -- 172.21.15.43:0/2775495406 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb80095f0 con 0x7f4bbc106c60 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc15ed700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4bbc100c20 0x7f4bbc19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc0dec700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4bbc1015c0 0x7f4bbc19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.634+0000 7f4bc1dee700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4bbc106c60 0x7f4bbc197650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4015000 tx=0x7f4bb40079d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.635+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400e040 con 0x7f4bbc106c60 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.635+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb4009c40 con 0x7f4bbc106c60 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.635+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4bb400bbf0 con 0x7f4bbc106c60 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.635+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4005040 con 0x7f4bbc106c60 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.635+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc1a4330 con 0x7f4bbc106c60 2024-02-05T07:34:51.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.635+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4bbc110400 con 0x7f4bbc106c60 2024-02-05T07:34:51.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.636+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f4bb401b450 con 0x7f4bbc106c60 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.636+0000 7f4bb27fc700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f4ba806dac0 0x7f4ba806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bc15ed700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f4ba806dac0 0x7f4ba806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f4bb408ed10 con 0x7f4bbc106c60 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bb27fc700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] conn(0x7f4ba8077820 0x7f4ba8079cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 --> [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ba807a390 con 0x7f4ba8077820 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4bb4092020 con 0x7f4bbc106c60 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bc15ed700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f4ba806dac0 0x7f4ba806ff80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8005fd0 tx=0x7f4bb8009310 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bc0dec700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] conn(0x7f4ba8077820 0x7f4ba8079cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.637+0000 7f4bc0dec700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] conn(0x7f4ba8077820 0x7f4ba8079cc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:51.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.638+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== osd.4 v2:172.21.15.50:6800/2018737057 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f4ba807a390 con 0x7f4ba8077820 2024-02-05T07:34:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.655+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 --> [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4bbc104c60 con 0x7f4ba8077820 2024-02-05T07:34:51.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.655+0000 7f4bb27fc700 1 -- 172.21.15.43:0/2775495406 <== osd.4 v2:172.21.15.50:6800/2018737057 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4bbc104c60 con 0x7f4ba8077820 2024-02-05T07:34:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] conn(0x7f4ba8077820 msgr2=0x7f4ba8079cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.50:6800/2018737057,v1:172.21.15.50:6801/2018737057] conn(0x7f4ba8077820 0x7f4ba8079cc0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f4ba806dac0 msgr2=0x7f4ba806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f4ba806dac0 0x7f4ba806ff80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4bb8005fd0 tx=0x7f4bb8009310 comp rx=0 tx=0).stop 2024-02-05T07:34:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4bbc106c60 msgr2=0x7f4bbc197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 --2- 172.21.15.43:0/2775495406 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4bbc106c60 0x7f4bbc197650 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4bb4015000 tx=0x7f4bb40079d0 comp rx=0 tx=0).stop 2024-02-05T07:34:51.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc1dee700 1 -- 172.21.15.43:0/2775495406 reap_dead start 2024-02-05T07:34:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 shutdown_connections 2024-02-05T07:34:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 >> 172.21.15.43:0/2775495406 conn(0x7f4bbc077b50 msgr2=0x7f4bbc0fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 shutdown_connections 2024-02-05T07:34:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:51.656+0000 7f4bc3851700 1 -- 172.21.15.43:0/2775495406 wait complete. 2024-02-05T07:34:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:34:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3816761021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94101680 msgr2=0x7f1a94101a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 --2- 172.21.15.43:0/3816761021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94101680 0x7f1a94101a60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1a900098f0 tx=0x7f1a90009c00 comp rx=0 tx=0).stop 2024-02-05T07:34:52.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3816761021 shutdown_connections 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 --2- 172.21.15.43:0/3816761021 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a94105fa0 0x7f1a9410db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 --2- 172.21.15.43:0/3816761021 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94101680 0x7f1a94101a60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 --2- 172.21.15.43:0/3816761021 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a94102030 0x7f1a941058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3816761021 >> 172.21.15.43:0/3816761021 conn(0x7f1a94077220 msgr2=0x7f1a940fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3816761021 shutdown_connections 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.920+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3816761021 wait complete. 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 Processor -- start 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 -- start start 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a94101680 0x7f1a9419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94102030 0x7f1a9419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a94105fa0 0x7f1a94197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94197e30 con 0x7f1a94102030 2024-02-05T07:34:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94197fa0 con 0x7f1a94105fa0 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a9b7ad700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94198160 con 0x7f1a94101680 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a99549700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a94101680 0x7f1a9419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a98d48700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94102030 0x7f1a9419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a99549700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a94101680 0x7f1a9419d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:58536/0 (socket says 172.21.15.43:58536) 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a99549700 1 -- 172.21.15.43:0/3752369086 learned_addr learned my addr 172.21.15.43:0/3752369086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.921+0000 7f1a99d4a700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a94105fa0 0x7f1a94197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a99549700 1 -- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a94105fa0 msgr2=0x7f1a94197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a99549700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a94105fa0 0x7f1a94197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a99549700 1 -- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94102030 msgr2=0x7f1a9419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a99549700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a94102030 0x7f1a9419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a99549700 1 -- 172.21.15.43:0/3752369086 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a840095f0 con 0x7f1a94101680 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a99d4a700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f1a94105fa0 0x7f1a94197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a99549700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a94101680 0x7f1a9419d560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a90011040 tx=0x7f1a9000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:52.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9001c070 con 0x7f1a94101680 2024-02-05T07:34:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1a9000bae0 con 0x7f1a94101680 2024-02-05T07:34:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a90004930 con 0x7f1a94101680 2024-02-05T07:34:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a90009520 con 0x7f1a94101680 2024-02-05T07:34:52.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.922+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a941a4310 con 0x7f1a94101680 2024-02-05T07:34:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.923+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1a941104b0 con 0x7f1a94101680 2024-02-05T07:34:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.924+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f1a90020070 con 0x7f1a94101680 2024-02-05T07:34:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.924+0000 7f1a8a7fc700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f1a8006db10 0x7f1a8006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.924+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f1a9008e4e0 con 0x7f1a94101680 2024-02-05T07:34:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.924+0000 7f1a98d48700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f1a8006db10 0x7f1a8006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:52.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.924+0000 7f1a8a7fc700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] conn(0x7f1a80077aa0 0x7f1a80079f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.924+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 --> [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1a8007a610 con 0x7f1a80077aa0 2024-02-05T07:34:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.924+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a90090bf0 con 0x7f1a94101680 2024-02-05T07:34:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.925+0000 7f1a99d4a700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] conn(0x7f1a80077aa0 0x7f1a80079f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.925+0000 7f1a98d48700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f1a8006db10 0x7f1a8006ffd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a94198df0 tx=0x7f1a84009310 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.925+0000 7f1a99d4a700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] conn(0x7f1a80077aa0 0x7f1a80079f40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:52.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.926+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== osd.7 v2:172.21.15.50:6824/1826641620 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1a8007a610 con 0x7f1a80077aa0 2024-02-05T07:34:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.942+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 --> [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1a94105180 con 0x7f1a80077aa0 2024-02-05T07:34:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a8a7fc700 1 -- 172.21.15.43:0/3752369086 <== osd.7 v2:172.21.15.50:6824/1826641620 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1a94105180 con 0x7f1a80077aa0 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] conn(0x7f1a80077aa0 msgr2=0x7f1a80079f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.50:6824/1826641620,v1:172.21.15.50:6825/1826641620] conn(0x7f1a80077aa0 0x7f1a80079f40 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f1a8006db10 msgr2=0x7f1a8006ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f1a8006db10 0x7f1a8006ffd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1a94198df0 tx=0x7f1a84009310 comp rx=0 tx=0).stop 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a94101680 msgr2=0x7f1a9419d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 --2- 172.21.15.43:0/3752369086 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f1a94101680 0x7f1a9419d560 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1a90011040 tx=0x7f1a9000fa00 comp rx=0 tx=0).stop 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a99d4a700 1 -- 172.21.15.43:0/3752369086 reap_dead start 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 shutdown_connections 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 >> 172.21.15.43:0/3752369086 conn(0x7f1a94077220 msgr2=0x7f1a941005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 shutdown_connections 2024-02-05T07:34:52.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:52.943+0000 7f1a9b7ad700 1 -- 172.21.15.43:0/3752369086 wait complete. 2024-02-05T07:34:53.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:53 smithi043 ceph-mon[29879]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-05T07:34:53.222 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:53 smithi043 ceph-mon[41444]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-05T07:34:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:53 smithi050 ceph-mon[33492]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-02-05T07:34:53.526 INFO:teuthology.orchestra.run.smithi043.stdout:111669149731 2024-02-05T07:34:53.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-05T07:34:54.238 INFO:teuthology.orchestra.run.smithi043.stdout:240518168591 2024-02-05T07:34:54.239 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 -- 172.21.15.43:0/4251345182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0041008c0 msgr2=0x7fd004100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 --2- 172.21.15.43:0/4251345182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0041008c0 0x7fd004100ca0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcff400b0a0 tx=0x7fcff400b3b0 comp rx=0 tx=0).stop 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 -- 172.21.15.43:0/4251345182 shutdown_connections 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 --2- 172.21.15.43:0/4251345182 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0041070e0 0x7fd00410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 --2- 172.21.15.43:0/4251345182 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0041008c0 0x7fd004100ca0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 --2- 172.21.15.43:0/4251345182 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd004101270 0x7fd0041069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 -- 172.21.15.43:0/4251345182 >> 172.21.15.43:0/4251345182 conn(0x7fd004075f60 msgr2=0x7fd004076370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:54.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 -- 172.21.15.43:0/4251345182 shutdown_connections 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.576+0000 7fd008aa6700 1 -- 172.21.15.43:0/4251345182 wait complete. 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 Processor -- start 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 -- start start 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0041008c0 0x7fd00419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd004101270 0x7fd00419db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0041070e0 0x7fd0041977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004197e50 con 0x7fd004101270 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd004197fc0 con 0x7fd0041008c0 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd008aa6700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd004198180 con 0x7fd0041070e0 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd004101270 0x7fd00419db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:54.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd002d9d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0041070e0 0x7fd0041977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd004101270 0x7fd00419db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54086/0 (socket says 172.21.15.43:54086) 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 -- 172.21.15.43:0/2018953120 learned_addr learned my addr 172.21.15.43:0/2018953120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd00259c700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0041008c0 0x7fd00419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 -- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0041070e0 msgr2=0x7fd0041977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0041070e0 0x7fd0041977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 -- 172.21.15.43:0/2018953120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0041008c0 msgr2=0x7fd00419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd0041008c0 0x7fd00419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd001d9b700 1 -- 172.21.15.43:0/2018953120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec009520 con 0x7fd004101270 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.577+0000 7fd002d9d700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd0041070e0 0x7fd0041977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.578+0000 7fd001d9b700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd004101270 0x7fd00419db60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009fd0 tx=0x7fcfec00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.578+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec00cbd0 con 0x7fd004101270 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.578+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fcfec00eea0 con 0x7fd004101270 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.578+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff4009d00 con 0x7fd004101270 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.578+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec003b70 con 0x7fd004101270 2024-02-05T07:34:54.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.578+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0041a4480 con 0x7fd004101270 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.579+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd004111630 con 0x7fd004101270 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.579+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fcfec003cf0 con 0x7fd004101270 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fcffb7fe700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fcff006dac0 0x7fcff006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fcfec014070 con 0x7fd004101270 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fd00259c700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fcff006dac0 0x7fcff006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fcffb7fe700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] conn(0x7fcff0077950 0x7fcff0079df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 --> [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcff007a4c0 con 0x7fcff0077950 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfec08e0d0 con 0x7fd004101270 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fd002d9d700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] conn(0x7fcff0077950 0x7fcff0079df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fd00259c700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fcff006dac0 0x7fcff006ff80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcff400b780 tx=0x7fcff4006210 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.580+0000 7fd002d9d700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] conn(0x7fcff0077950 0x7fcff0079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:34:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.581+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== osd.0 v2:172.21.15.43:6802/2715902880 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcff007a4c0 con 0x7fcff0077950 2024-02-05T07:34:54.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 --> [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0040623b0 con 0x7fcff0077950 2024-02-05T07:34:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fcffb7fe700 1 -- 172.21.15.43:0/2018953120 <== osd.0 v2:172.21.15.43:6802/2715902880 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd0040623b0 con 0x7fcff0077950 2024-02-05T07:34:54.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] conn(0x7fcff0077950 msgr2=0x7fcff0079df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6802/2715902880,v1:172.21.15.43:6803/2715902880] conn(0x7fcff0077950 0x7fcff0079df0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fcff006dac0 msgr2=0x7fcff006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fcff006dac0 0x7fcff006ff80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcff400b780 tx=0x7fcff4006210 comp rx=0 tx=0).stop 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd004101270 msgr2=0x7fd00419db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 --2- 172.21.15.43:0/2018953120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd004101270 0x7fd00419db60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009fd0 tx=0x7fcfec00ed10 comp rx=0 tx=0).stop 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd002d9d700 1 -- 172.21.15.43:0/2018953120 reap_dead start 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 shutdown_connections 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 >> 172.21.15.43:0/2018953120 conn(0x7fd004075f60 msgr2=0x7fd004108280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 shutdown_connections 2024-02-05T07:34:54.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:34:54.598+0000 7fd008aa6700 1 -- 172.21.15.43:0/2018953120 wait complete. 2024-02-05T07:34:55.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:55 smithi043 ceph-mon[29879]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:55.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:55 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:34:55.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:55 smithi043 ceph-mon[41444]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:55.160 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:55 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:34:55.184 INFO:teuthology.orchestra.run.smithi043.stdout:34359738419 2024-02-05T07:34:55.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-05T07:34:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:55 smithi050 ceph-mon[33492]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:55 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:34:55.469 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:56.003 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:56.004 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:56.408 INFO:teuthology.orchestra.run.smithi043.stdout:137438953504 2024-02-05T07:34:56.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-05T07:34:56.836 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:57 smithi043 ceph-mon[29879]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:57 smithi043 ceph-mon[41444]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:57.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:57 smithi050 ceph-mon[33492]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:58.684 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:58.690 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:34:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:34:59 smithi050 ceph-mon[33492]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:59.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:34:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:34:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:34:59.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:34:59 smithi043 ceph-mon[41444]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:34:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:34:59 smithi043 ceph-mon[29879]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-05T07:35:00.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 -- 172.21.15.43:0/1889912462 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c106cb0 msgr2=0x7f627c107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 --2- 172.21.15.43:0/1889912462 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c106cb0 0x7f627c107090 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f626c009990 tx=0x7f626c009ca0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 -- 172.21.15.43:0/1889912462 shutdown_connections 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 --2- 172.21.15.43:0/1889912462 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c1015c0 0x7f627c10da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 --2- 172.21.15.43:0/1889912462 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c106cb0 0x7f627c107090 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 --2- 172.21.15.43:0/1889912462 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f627c100c00 0x7f627c101080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 -- 172.21.15.43:0/1889912462 >> 172.21.15.43:0/1889912462 conn(0x7f627c077c30 msgr2=0x7f627c078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 -- 172.21.15.43:0/1889912462 shutdown_connections 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.140+0000 7f6281c28700 1 -- 172.21.15.43:0/1889912462 wait complete. 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 Processor -- start 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 -- start start 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c100c00 0x7f627c1965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:00.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c1015c0 0x7f627c196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f627c106cb0 0x7f627c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c19b4c0 con 0x7f627c106cb0 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627c19b630 con 0x7f627c100c00 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f6281c28700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627c19b7f0 con 0x7f627c1015c0 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f627affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c1015c0 0x7f627c196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f627bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f627c106cb0 0x7f627c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f627bfff700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f627c106cb0 0x7f627c19aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54096/0 (socket says 172.21.15.43:54096) 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f627affd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c1015c0 0x7f627c196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:58552/0 (socket says 172.21.15.43:58552) 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f627bfff700 1 -- 172.21.15.43:0/935956399 learned_addr learned my addr 172.21.15.43:0/935956399 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.141+0000 7f627b7fe700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c100c00 0x7f627c1965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f627affd700 1 -- 172.21.15.43:0/935956399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c100c00 msgr2=0x7f627c1965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f627affd700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c100c00 0x7f627c1965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f627affd700 1 -- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f627c106cb0 msgr2=0x7f627c19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f627affd700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f627c106cb0 0x7f627c19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f627affd700 1 -- 172.21.15.43:0/935956399 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6270009520 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f627affd700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c1015c0 0x7f627c196b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9f0 tx=0x7f627000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f6278ff9700 1 -- 172.21.15.43:0/935956399 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627000cb70 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626c0095f0 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f6278ff9700 1 -- 172.21.15.43:0/935956399 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f627000ee90 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f6278ff9700 1 -- 172.21.15.43:0/935956399 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6270003b30 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.142+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627c06a440 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.143+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627c0689e0 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.144+0000 7f6278ff9700 1 -- 172.21.15.43:0/935956399 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f6270003d70 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.144+0000 7f6278ff9700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f626406dac0 0x7f626406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.144+0000 7f6278ff9700 1 -- 172.21.15.43:0/935956399 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f6270014070 con 0x7f627c1015c0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.144+0000 7f627b7fe700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f626406dac0 0x7f626406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:00.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.144+0000 7f627b7fe700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f626406dac0 0x7f626406ff80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f626c006010 tx=0x7f626c00b260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:00.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.149+0000 7f6278ff9700 1 -- 172.21.15.43:0/935956399 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6270056fe0 con 0x7f627c1015c0 2024-02-05T07:35:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.356+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f627c19b9b0 con 0x7f627c1015c0 2024-02-05T07:35:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.356+0000 7f6278ff9700 1 -- 172.21.15.43:0/935956399 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f627005a6a0 con 0x7f627c1015c0 2024-02-05T07:35:00.358 INFO:teuthology.orchestra.run.smithi043.stdout:77309411371 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.358+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f626406dac0 msgr2=0x7f626406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.358+0000 7f6281c28700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f626406dac0 0x7f626406ff80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f626c006010 tx=0x7f626c00b260 comp rx=0 tx=0).stop 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.358+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c1015c0 msgr2=0x7f627c196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.358+0000 7f6281c28700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c1015c0 0x7f627c196b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9f0 tx=0x7f627000ed00 comp rx=0 tx=0).stop 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 shutdown_connections 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f627c106cb0 0x7f627c19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f627c100c00 0x7f627c1965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f626406dac0 0x7f626406ff80 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 --2- 172.21.15.43:0/935956399 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f627c1015c0 0x7f627c196b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 >> 172.21.15.43:0/935956399 conn(0x7f627c077c30 msgr2=0x7f627c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 shutdown_connections 2024-02-05T07:35:00.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:00.359+0000 7f6281c28700 1 -- 172.21.15.43:0/935956399 wait complete. 2024-02-05T07:35:01.069 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:35:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:35:01.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 -- 172.21.15.43:0/1708926976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe388107d70 msgr2=0x7fe388108150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:01.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 --2- 172.21.15.43:0/1708926976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe388107d70 0x7fe388108150 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe3780099c0 tx=0x7fe378009cd0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 -- 172.21.15.43:0/1708926976 shutdown_connections 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 --2- 172.21.15.43:0/1708926976 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe38806a370 0x7fe38810ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 --2- 172.21.15.43:0/1708926976 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe388107d70 0x7fe388108150 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 --2- 172.21.15.43:0/1708926976 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe3880699d0 0x7fe388069e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 -- 172.21.15.43:0/1708926976 >> 172.21.15.43:0/1708926976 conn(0x7fe388077b50 msgr2=0x7fe388077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 -- 172.21.15.43:0/1708926976 shutdown_connections 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.003+0000 7fe38e250700 1 -- 172.21.15.43:0/1708926976 wait complete. 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 Processor -- start 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 -- start start 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3880699d0 0x7fe388100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe38806a370 0x7fe388101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe388107d70 0x7fe388101770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3881a5b40 con 0x7fe3880699d0 2024-02-05T07:35:01.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3881a5cb0 con 0x7fe38806a370 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe38e250700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3881a5e70 con 0x7fe388107d70 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3880699d0 0x7fe388100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3880699d0 0x7fe388100cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54106/0 (socket says 172.21.15.43:54106) 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 -- 172.21.15.43:0/2092044174 learned_addr learned my addr 172.21.15.43:0/2092044174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe387fff700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe388107d70 0x7fe388101770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe386ffd700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe38806a370 0x7fe388101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 -- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe388107d70 msgr2=0x7fe388101770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe388107d70 0x7fe388101770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 -- 172.21.15.43:0/2092044174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe38806a370 msgr2=0x7fe388101230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe38806a370 0x7fe388101230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 -- 172.21.15.43:0/2092044174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c009520 con 0x7fe3880699d0 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.004+0000 7fe3877fe700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3880699d0 0x7fe388100cf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe378006010 tx=0x7fe37800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.005+0000 7fe384ff9700 1 -- 172.21.15.43:0/2092044174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37801d070 con 0x7fe3880699d0 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.005+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3780095f0 con 0x7fe3880699d0 2024-02-05T07:35:01.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.005+0000 7fe384ff9700 1 -- 172.21.15.43:0/2092044174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe378003b00 con 0x7fe3880699d0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.005+0000 7fe384ff9700 1 -- 172.21.15.43:0/2092044174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800f950 con 0x7fe3880699d0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.005+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a63d0 con 0x7fe3880699d0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.006+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38810a180 con 0x7fe3880699d0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.006+0000 7fe384ff9700 1 -- 172.21.15.43:0/2092044174 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fe378003680 con 0x7fe3880699d0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.007+0000 7fe384ff9700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe37006dac0 0x7fe37006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.007+0000 7fe384ff9700 1 -- 172.21.15.43:0/2092044174 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fe37808e500 con 0x7fe3880699d0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.007+0000 7fe386ffd700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe37006dac0 0x7fe37006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.007+0000 7fe386ffd700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe37006dac0 0x7fe37006ff80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe388078370 tx=0x7fe37c009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:01.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.012+0000 7fe384ff9700 1 -- 172.21.15.43:0/2092044174 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe3780579c0 con 0x7fe3880699d0 2024-02-05T07:35:01.291 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:01 smithi043 ceph-mon[41444]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:01.292 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/935956399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T07:35:01.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:01 smithi043 ceph-mon[29879]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:01.292 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/935956399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T07:35:01.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.222+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe3881021d0 con 0x7fe3880699d0 2024-02-05T07:35:01.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.222+0000 7fe384ff9700 1 -- 172.21.15.43:0/2092044174 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe37805b080 con 0x7fe3880699d0 2024-02-05T07:35:01.293 INFO:teuthology.orchestra.run.smithi043.stdout:55834574896 2024-02-05T07:35:01.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.224+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe37006dac0 msgr2=0x7fe37006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:01.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe37006dac0 0x7fe37006ff80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fe388078370 tx=0x7fe37c009260 comp rx=0 tx=0).stop 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3880699d0 msgr2=0x7fe388100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3880699d0 0x7fe388100cf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe378006010 tx=0x7fe37800bb00 comp rx=0 tx=0).stop 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 shutdown_connections 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe37006dac0 0x7fe37006ff80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe388107d70 0x7fe388101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3880699d0 0x7fe388100cf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 --2- 172.21.15.43:0/2092044174 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe38806a370 0x7fe388101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 >> 172.21.15.43:0/2092044174 conn(0x7fe388077b50 msgr2=0x7fe3880ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 shutdown_connections 2024-02-05T07:35:01.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:01.225+0000 7fe38e250700 1 -- 172.21.15.43:0/2092044174 wait complete. 2024-02-05T07:35:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:01 smithi050 ceph-mon[33492]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/935956399' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-05T07:35:01.549 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411371 for osd.2 2024-02-05T07:35:01.549 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:02.156 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:02.156 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2092044174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T07:35:02.156 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:02.156 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:02.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:02.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2092044174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T07:35:02.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:02.157 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:02 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2092044174' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-05T07:35:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:02 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:02 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:35:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:35:03.372 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:03 smithi043 ceph-mon[29879]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:03.372 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:03 smithi043 ceph-mon[41444]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:03 smithi050 ceph-mon[33492]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 -- 172.21.15.43:0/3283782869 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54102c60 msgr2=0x7fec54103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 --2- 172.21.15.43:0/3283782869 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54102c60 0x7fec54103040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fec500098f0 tx=0x7fec50009c00 comp rx=0 tx=0).stop 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 -- 172.21.15.43:0/3283782869 shutdown_connections 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 --2- 172.21.15.43:0/3283782869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec54100ea0 0x7fec54110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 --2- 172.21.15.43:0/3283782869 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54102c60 0x7fec54103040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 --2- 172.21.15.43:0/3283782869 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54100310 0x7fec54100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 -- 172.21.15.43:0/3283782869 >> 172.21.15.43:0/3283782869 conn(0x7fec54077220 msgr2=0x7fec540fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 -- 172.21.15.43:0/3283782869 shutdown_connections 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.140+0000 7fec5be37700 1 -- 172.21.15.43:0/3283782869 wait complete. 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 Processor -- start 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 -- start start 2024-02-05T07:35:04.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec54100310 0x7fec5419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54100ea0 0x7fec5419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54102c60 0x7fec54199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec54199f70 con 0x7fec54100310 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5419a0b0 con 0x7fec54102c60 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5be37700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5419a270 con 0x7fec54100ea0 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec59bd3700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec54100310 0x7fec5419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec593d2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54100ea0 0x7fec5419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec593d2700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54100ea0 0x7fec5419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54346/0 (socket says 172.21.15.43:54346) 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec593d2700 1 -- 172.21.15.43:0/987924799 learned_addr learned my addr 172.21.15.43:0/987924799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec5a3d4700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54102c60 0x7fec54199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec593d2700 1 -- 172.21.15.43:0/987924799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54102c60 msgr2=0x7fec54199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec593d2700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54102c60 0x7fec54199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec593d2700 1 -- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec54100310 msgr2=0x7fec5419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:04.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.141+0000 7fec593d2700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec54100310 0x7fec5419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec593d2700 1 -- 172.21.15.43:0/987924799 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c009d00 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec59bd3700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec54100310 0x7fec5419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec593d2700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54100ea0 0x7fec5419fc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec4400e9a0 tx=0x7fec4400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec4affd700 1 -- 172.21.15.43:0/987924799 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44009790 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec4affd700 1 -- 172.21.15.43:0/987924799 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fec4400ce90 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec50009520 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec4affd700 1 -- 172.21.15.43:0/987924799 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44018420 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.142+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec541a6620 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.143+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec54101320 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.144+0000 7fec4affd700 1 -- 172.21.15.43:0/987924799 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fec4400ca10 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.144+0000 7fec4affd700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fec4006dac0 0x7fec4006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.144+0000 7fec4affd700 1 -- 172.21.15.43:0/987924799 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fec44014070 con 0x7fec54100ea0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.144+0000 7fec59bd3700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fec4006dac0 0x7fec4006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.145+0000 7fec59bd3700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fec4006dac0 0x7fec4006ff80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fec5419aa50 tx=0x7fec50019040 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.149+0000 7fec4affd700 1 -- 172.21.15.43:0/987924799 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec44052d20 con 0x7fec54100ea0 2024-02-05T07:35:04.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi043 to 3761M 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi043:/etc/ceph/ceph.conf 2024-02-05T07:35:04.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi043 to 3761M 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi043:/etc/ceph/ceph.conf 2024-02-05T07:35:04.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.369+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fec5419ae40 con 0x7fec54100ea0 2024-02-05T07:35:04.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.370+0000 7fec4affd700 1 -- 172.21.15.43:0/987924799 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec44056ee0 con 0x7fec54100ea0 2024-02-05T07:35:04.371 INFO:teuthology.orchestra.run.smithi043.stdout:206158430230 2024-02-05T07:35:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fec4006dac0 msgr2=0x7fec4006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fec4006dac0 0x7fec4006ff80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fec5419aa50 tx=0x7fec50019040 comp rx=0 tx=0).stop 2024-02-05T07:35:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54100ea0 msgr2=0x7fec5419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54100ea0 0x7fec5419fc90 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fec4400e9a0 tx=0x7fec4400ecb0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 shutdown_connections 2024-02-05T07:35:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fec54102c60 0x7fec54199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec54100310 0x7fec5419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fec4006dac0 0x7fec4006ff80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 --2- 172.21.15.43:0/987924799 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fec54100ea0 0x7fec5419fc90 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:04.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 >> 172.21.15.43:0/987924799 conn(0x7fec54077220 msgr2=0x7fec540fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:04.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 shutdown_connections 2024-02-05T07:35:04.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:04.372+0000 7fec5be37700 1 -- 172.21.15.43:0/987924799 wait complete. 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi043 to 3761M 2024-02-05T07:35:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi043:/etc/ceph/ceph.conf 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi050:/etc/ceph/ceph.conf 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: Updating smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:04.575 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:05.320 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:05 smithi043 ceph-mon[29879]: Deploying daemon alertmanager.a on smithi043 2024-02-05T07:35:05.320 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/987924799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-05T07:35:05.321 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:05 smithi043 ceph-mon[41444]: Deploying daemon alertmanager.a on smithi043 2024-02-05T07:35:05.321 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/987924799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-05T07:35:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:05 smithi050 ceph-mon[33492]: Deploying daemon alertmanager.a on smithi043 2024-02-05T07:35:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/987924799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-05T07:35:06.087 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430227 got 206158430230 for osd.6 2024-02-05T07:35:06.087 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:06 smithi043 ceph-mon[29879]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:35:06.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:06 smithi043 ceph-mon[41444]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:35:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:06 smithi050 ceph-mon[33492]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:35:07.336 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574894 got 55834574896 for osd.1 2024-02-05T07:35:07.336 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:08.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 -- 172.21.15.43:0/63220999 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58101270 msgr2=0x7ffa58105060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 --2- 172.21.15.43:0/63220999 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58101270 0x7ffa58105060 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009990 tx=0x7ffa4c009ca0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 -- 172.21.15.43:0/63220999 shutdown_connections 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 --2- 172.21.15.43:0/63220999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58105790 0x7ffa5810d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 --2- 172.21.15.43:0/63220999 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa581008c0 0x7ffa58100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 --2- 172.21.15.43:0/63220999 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58101270 0x7ffa58105060 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 -- 172.21.15.43:0/63220999 >> 172.21.15.43:0/63220999 conn(0x7ffa58075f80 msgr2=0x7ffa58076390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 -- 172.21.15.43:0/63220999 shutdown_connections 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 -- 172.21.15.43:0/63220999 wait complete. 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.637+0000 7ffa5df84700 1 Processor -- start 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa5df84700 1 -- start start 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa5df84700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa581008c0 0x7ffa5819d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:08.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa5df84700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58101270 0x7ffa5819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa5df84700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58105790 0x7ffa58197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa5df84700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58197c90 con 0x7ffa58101270 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa5df84700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58197dd0 con 0x7ffa581008c0 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa5df84700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa58197f90 con 0x7ffa58105790 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58101270 0x7ffa5819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa57fff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58105790 0x7ffa58197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58101270 0x7ffa5819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41210/0 (socket says 172.21.15.43:41210) 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 -- 172.21.15.43:0/3212673521 learned_addr learned my addr 172.21.15.43:0/3212673521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa577fe700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa581008c0 0x7ffa5819d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 -- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58105790 msgr2=0x7ffa58197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58105790 0x7ffa58197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 -- 172.21.15.43:0/3212673521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa581008c0 msgr2=0x7ffa5819d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa581008c0 0x7ffa5819d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 -- 172.21.15.43:0/3212673521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5000ad50 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa57fff700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58105790 0x7ffa58197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa577fe700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa581008c0 0x7ffa5819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.638+0000 7ffa56ffd700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58101270 0x7ffa5819d960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c000c00 tx=0x7ffa4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.639+0000 7ffa54ff9700 1 -- 172.21.15.43:0/3212673521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c01d070 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.639+0000 7ffa54ff9700 1 -- 172.21.15.43:0/3212673521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffa4c021410 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.639+0000 7ffa54ff9700 1 -- 172.21.15.43:0/3212673521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0173f0 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.639+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c0095f0 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.639+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa58198490 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.640+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa58108990 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.640+0000 7ffa54ff9700 1 -- 172.21.15.43:0/3212673521 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7ffa4c00fa90 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.640+0000 7ffa54ff9700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffa4006dac0 0x7ffa4006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.641+0000 7ffa577fe700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffa4006dac0 0x7ffa4006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.641+0000 7ffa54ff9700 1 -- 172.21.15.43:0/3212673521 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7ffa4c08e9d0 con 0x7ffa58101270 2024-02-05T07:35:08.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.641+0000 7ffa577fe700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffa4006dac0 0x7ffa4006ff80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa48005fd0 tx=0x7ffa48005ee0 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.645+0000 7ffa54ff9700 1 -- 172.21.15.43:0/3212673521 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffa4c057e90 con 0x7ffa58101270 2024-02-05T07:35:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.856+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7ffa58198b10 con 0x7ffa58101270 2024-02-05T07:35:08.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.856+0000 7ffa54ff9700 1 -- 172.21.15.43:0/3212673521 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa4c00fd40 con 0x7ffa58101270 2024-02-05T07:35:08.861 INFO:teuthology.orchestra.run.smithi043.stdout:171798691869 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffa4006dac0 msgr2=0x7ffa4006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffa4006dac0 0x7ffa4006ff80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffa48005fd0 tx=0x7ffa48005ee0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58101270 msgr2=0x7ffa5819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58101270 0x7ffa5819d960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c000c00 tx=0x7ffa4c004730 comp rx=0 tx=0).stop 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 shutdown_connections 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffa4006dac0 0x7ffa4006ff80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffa58105790 0x7ffa58197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffa581008c0 0x7ffa5819d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 --2- 172.21.15.43:0/3212673521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffa58101270 0x7ffa5819d960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:08.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.858+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 >> 172.21.15.43:0/3212673521 conn(0x7ffa58075f80 msgr2=0x7ffa580feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.859+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 shutdown_connections 2024-02-05T07:35:08.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:08.859+0000 7ffa5df84700 1 -- 172.21.15.43:0/3212673521 wait complete. 2024-02-05T07:35:08.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:08 smithi043 ceph-mon[29879]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:08.958 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:08 smithi043 ceph-mon[41444]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:09.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:08 smithi050 ceph-mon[33492]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 -- 172.21.15.43:0/3937368826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900107040 msgr2=0x7fd90010ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 --2- 172.21.15.43:0/3937368826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900107040 0x7fd90010ec10 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009990 tx=0x7fd8f8009ca0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 -- 172.21.15.43:0/3937368826 shutdown_connections 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 --2- 172.21.15.43:0/3937368826 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900107040 0x7fd90010ec10 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 --2- 172.21.15.43:0/3937368826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd900101f60 0x7fd900102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 --2- 172.21.15.43:0/3937368826 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd900102910 0x7fd900106960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 -- 172.21.15.43:0/3937368826 >> 172.21.15.43:0/3937368826 conn(0x7fd9000fd7d0 msgr2=0x7fd9000ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 -- 172.21.15.43:0/3937368826 shutdown_connections 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.061+0000 7fd907c48700 1 -- 172.21.15.43:0/3937368826 wait complete. 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 Processor -- start 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 -- start start 2024-02-05T07:35:09.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900101f60 0x7fd900074020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd900102910 0x7fd900074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd900107040 0x7fd900074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001a5ab0 con 0x7fd900107040 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001a5c20 con 0x7fd900102910 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd907c48700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9001a5de0 con 0x7fd900101f60 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd9059e4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900101f60 0x7fd900074020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd9059e4700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900101f60 0x7fd900074020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54380/0 (socket says 172.21.15.43:54380) 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd9059e4700 1 -- 172.21.15.43:0/651000779 learned_addr learned my addr 172.21.15.43:0/651000779 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd9061e5700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd900107040 0x7fd900074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd9051e3700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd900102910 0x7fd900074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd9059e4700 1 -- 172.21.15.43:0/651000779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd900102910 msgr2=0x7fd900074560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.062+0000 7fd9059e4700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd900102910 0x7fd900074560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd9059e4700 1 -- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd900107040 msgr2=0x7fd900074aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd9059e4700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd900107040 0x7fd900074aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd9059e4700 1 -- 172.21.15.43:0/651000779 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f0009c40 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd9061e5700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd900107040 0x7fd900074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd9059e4700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900101f60 0x7fd900074020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c370 tx=0x7fd8f000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd8feffd700 1 -- 172.21.15.43:0/651000779 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0017070 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f80095f0 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd8feffd700 1 -- 172.21.15.43:0/651000779 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd8f000f040 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd8feffd700 1 -- 172.21.15.43:0/651000779 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000e050 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.063+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9001a63d0 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.064+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90010a270 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.064+0000 7fd8feffd700 1 -- 172.21.15.43:0/651000779 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fd8f0003a00 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.065+0000 7fd8feffd700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd8e806dac0 0x7fd8e806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.065+0000 7fd8feffd700 1 -- 172.21.15.43:0/651000779 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fd8f008d280 con 0x7fd900101f60 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.065+0000 7fd9051e3700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd8e806dac0 0x7fd8e806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.065+0000 7fd9051e3700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd8e806dac0 0x7fd8e806ff80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd9000feb60 tx=0x7fd8f400a380 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:09.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.069+0000 7fd8feffd700 1 -- 172.21.15.43:0/651000779 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8f0056690 con 0x7fd900101f60 2024-02-05T07:35:09.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.276+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd900075550 con 0x7fd900101f60 2024-02-05T07:35:09.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.276+0000 7fd8feffd700 1 -- 172.21.15.43:0/651000779 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd8f0059d50 con 0x7fd900101f60 2024-02-05T07:35:09.277 INFO:teuthology.orchestra.run.smithi043.stdout:34359738422 2024-02-05T07:35:09.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd8e806dac0 msgr2=0x7fd8e806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd8e806dac0 0x7fd8e806ff80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd9000feb60 tx=0x7fd8f400a380 comp rx=0 tx=0).stop 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900101f60 msgr2=0x7fd900074020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900101f60 0x7fd900074020 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c370 tx=0x7fd8f000c680 comp rx=0 tx=0).stop 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 shutdown_connections 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd900107040 0x7fd900074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd8e806dac0 0x7fd8e806ff80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd900101f60 0x7fd900074020 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 --2- 172.21.15.43:0/651000779 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd900102910 0x7fd900074560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.279+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 >> 172.21.15.43:0/651000779 conn(0x7fd9000fd7d0 msgr2=0x7fd9000ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.280+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 shutdown_connections 2024-02-05T07:35:09.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.280+0000 7fd907c48700 1 -- 172.21.15.43:0/651000779 wait complete. 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3048598761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 msgr2=0x7ff29c069e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3048598761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 0x7ff29c069e30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff290009920 tx=0x7ff290009c30 comp rx=0 tx=0).stop 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3048598761 shutdown_connections 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3048598761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 0x7ff29c10ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3048598761 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff29c107d70 0x7ff29c108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3048598761 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 0x7ff29c069e30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3048598761 >> 172.21.15.43:0/3048598761 conn(0x7ff29c077b50 msgr2=0x7ff29c077f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:09.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3048598761 shutdown_connections 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.290+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3048598761 wait complete. 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 Processor -- start 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 -- start start 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 0x7ff29c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 0x7ff29c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff29c107d70 0x7ff29c1975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c197c00 con 0x7ff29c06a370 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c197d40 con 0x7ff29c107d70 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a47c1700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29c197f00 con 0x7ff29c0699d0 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a255d700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 0x7ff29c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a1d5c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 0x7ff29c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a1d5c700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 0x7ff29c19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41226/0 (socket says 172.21.15.43:41226) 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.291+0000 7ff2a1d5c700 1 -- 172.21.15.43:0/3325291582 learned_addr learned my addr 172.21.15.43:0/3325291582 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:09.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a2d5e700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff29c107d70 0x7ff29c1975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a1d5c700 1 -- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 msgr2=0x7ff29c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a1d5c700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 0x7ff29c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a1d5c700 1 -- 172.21.15.43:0/3325291582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff29c107d70 msgr2=0x7ff29c1975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a1d5c700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff29c107d70 0x7ff29c1975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a1d5c700 1 -- 172.21.15.43:0/3325291582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff28c009d00 con 0x7ff29c06a370 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a2d5e700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff29c107d70 0x7ff29c1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a1d5c700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 0x7ff29c19d920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff290009230 tx=0x7ff29000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff29b7fe700 1 -- 172.21.15.43:0/3325291582 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29001c070 con 0x7ff29c06a370 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff29b7fe700 1 -- 172.21.15.43:0/3325291582 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff29000bae0 con 0x7ff29c06a370 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff290009520 con 0x7ff29c06a370 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff29b7fe700 1 -- 172.21.15.43:0/3325291582 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff290004930 con 0x7ff29c06a370 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.292+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29c1983a0 con 0x7ff29c06a370 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.293+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29c10a290 con 0x7ff29c06a370 2024-02-05T07:35:09.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.294+0000 7ff29b7fe700 1 -- 172.21.15.43:0/3325291582 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7ff290020070 con 0x7ff29c06a370 2024-02-05T07:35:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.294+0000 7ff29b7fe700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff28406dac0 0x7ff28406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.294+0000 7ff29b7fe700 1 -- 172.21.15.43:0/3325291582 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7ff29008e680 con 0x7ff29c06a370 2024-02-05T07:35:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.294+0000 7ff2a255d700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff28406dac0 0x7ff28406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.295+0000 7ff2a255d700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff28406dac0 0x7ff28406ff80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff29400abd0 tx=0x7ff29400a380 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:09.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.299+0000 7ff29b7fe700 1 -- 172.21.15.43:0/3325291582 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff290057a90 con 0x7ff29c06a370 2024-02-05T07:35:09.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.503+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff29c198ad0 con 0x7ff29c06a370 2024-02-05T07:35:09.505 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:35:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:35:09.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.504+0000 7ff29b7fe700 1 -- 172.21.15.43:0/3325291582 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff29005b150 con 0x7ff29c06a370 2024-02-05T07:35:09.505 INFO:teuthology.orchestra.run.smithi043.stdout:240518168594 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff28406dac0 msgr2=0x7ff28406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff28406dac0 0x7ff28406ff80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff29400abd0 tx=0x7ff29400a380 comp rx=0 tx=0).stop 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 msgr2=0x7ff29c19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 0x7ff29c19d920 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff290009230 tx=0x7ff29000fa00 comp rx=0 tx=0).stop 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 shutdown_connections 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ff29c107d70 0x7ff29c1975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ff28406dac0 0x7ff28406ff80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ff29c0699d0 0x7ff29c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 --2- 172.21.15.43:0/3325291582 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff29c06a370 0x7ff29c19d920 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 >> 172.21.15.43:0/3325291582 conn(0x7ff29c077b50 msgr2=0x7ff29c0ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 shutdown_connections 2024-02-05T07:35:09.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:09.506+0000 7ff2a47c1700 1 -- 172.21.15.43:0/3325291582 wait complete. 2024-02-05T07:35:09.759 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3212673521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-05T07:35:09.759 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/651000779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T07:35:09.759 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3325291582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-05T07:35:09.759 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3212673521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-05T07:35:09.759 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/651000779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T07:35:09.760 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3325291582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-05T07:35:10.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3212673521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-05T07:35:10.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/651000779' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-05T07:35:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3325291582' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-05T07:35:10.219 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738422 for osd.0 2024-02-05T07:35:10.220 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:10.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 -- 172.21.15.43:0/3497191323 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 msgr2=0x7ffb0c10d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:10.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 --2- 172.21.15.43:0/3497191323 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c10d2c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b0a0 tx=0x7ffb0400b3b0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 -- 172.21.15.43:0/3497191323 shutdown_connections 2024-02-05T07:35:10.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 --2- 172.21.15.43:0/3497191323 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c10d2c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 --2- 172.21.15.43:0/3497191323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb0c1008b0 0x7ffb0c100c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 --2- 172.21.15.43:0/3497191323 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c101260 0x7ffb0c105050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 -- 172.21.15.43:0/3497191323 >> 172.21.15.43:0/3497191323 conn(0x7ffb0c075f50 msgr2=0x7ffb0c076360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 -- 172.21.15.43:0/3497191323 shutdown_connections 2024-02-05T07:35:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 -- 172.21.15.43:0/3497191323 wait complete. 2024-02-05T07:35:10.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 Processor -- start 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.584+0000 7ffb11daa700 1 -- start start 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb11daa700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c1008b0 0x7ffb0c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb11daa700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb0c101260 0x7ffb0c198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb11daa700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c1a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb11daa700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c199410 con 0x7ffb0c101260 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb11daa700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1a3bd0 con 0x7ffb0c1008b0 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb11daa700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c1a3d10 con 0x7ffb0c105780 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c1a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0affd700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb0c101260 0x7ffb0c198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c1a3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:54420/0 (socket says 172.21.15.43:54420) 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 -- 172.21.15.43:0/2943487609 learned_addr learned my addr 172.21.15.43:0/2943487609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:10.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0b7fe700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c1008b0 0x7ffb0c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 -- 172.21.15.43:0/2943487609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c1008b0 msgr2=0x7ffb0c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c1008b0 0x7ffb0c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 -- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb0c101260 msgr2=0x7ffb0c198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb0c101260 0x7ffb0c198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 -- 172.21.15.43:0/2943487609 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc009520 con 0x7ffb0c105780 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0b7fe700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c1008b0 0x7ffb0c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb0bfff700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c1a3690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b780 tx=0x7ffb0400bd00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.585+0000 7ffb08ff9700 1 -- 172.21.15.43:0/2943487609 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e040 con 0x7ffb0c105780 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.586+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb04009d00 con 0x7ffb0c105780 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.586+0000 7ffb08ff9700 1 -- 172.21.15.43:0/2943487609 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffb04004e40 con 0x7ffb0c105780 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.586+0000 7ffb08ff9700 1 -- 172.21.15.43:0/2943487609 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04005040 con 0x7ffb0c105780 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.586+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c1a4270 con 0x7ffb0c105780 2024-02-05T07:35:10.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.586+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c0689e0 con 0x7ffb0c105780 2024-02-05T07:35:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.587+0000 7ffb08ff9700 1 -- 172.21.15.43:0/2943487609 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7ffb0401d470 con 0x7ffb0c105780 2024-02-05T07:35:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.587+0000 7ffb08ff9700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffaf406dac0 0x7ffaf406ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.588+0000 7ffb08ff9700 1 -- 172.21.15.43:0/2943487609 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7ffb0408efc0 con 0x7ffb0c105780 2024-02-05T07:35:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.588+0000 7ffb0b7fe700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffaf406dac0 0x7ffaf406ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:10.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.588+0000 7ffb0b7fe700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffaf406dac0 0x7ffaf406ff80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009d70 tx=0x7ffafc009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:10.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.592+0000 7ffb08ff9700 1 -- 172.21.15.43:0/2943487609 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb040c2d20 con 0x7ffb0c105780 2024-02-05T07:35:10.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.798+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffb0c10f940 con 0x7ffb0c105780 2024-02-05T07:35:10.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:10 smithi043 ceph-mon[29879]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:10.800 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:10 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:10.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:10 smithi043 ceph-mon[41444]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:10.800 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:10 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:10.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.799+0000 7ffb08ff9700 1 -- 172.21.15.43:0/2943487609 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb040583d0 con 0x7ffb0c105780 2024-02-05T07:35:10.801 INFO:teuthology.orchestra.run.smithi043.stdout:111669149735 2024-02-05T07:35:10.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffaf406dac0 msgr2=0x7ffaf406ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffaf406dac0 0x7ffaf406ff80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ffafc009d70 tx=0x7ffafc009260 comp rx=0 tx=0).stop 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 msgr2=0x7ffb0c1a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c1a3690 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffb0400b780 tx=0x7ffb0400bd00 comp rx=0 tx=0).stop 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 shutdown_connections 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7ffaf406dac0 0x7ffaf406ff80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7ffb0c105780 0x7ffb0c1a3690 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7ffb0c1008b0 0x7ffb0c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 --2- 172.21.15.43:0/2943487609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb0c101260 0x7ffb0c198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 >> 172.21.15.43:0/2943487609 conn(0x7ffb0c075f50 msgr2=0x7ffb0c103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 shutdown_connections 2024-02-05T07:35:10.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:10.801+0000 7ffb11daa700 1 -- 172.21.15.43:0/2943487609 wait complete. 2024-02-05T07:35:11.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:10 smithi050 ceph-mon[33492]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:10 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:11.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:35:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:35:11.116 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691865 got 171798691869 for osd.5 2024-02-05T07:35:11.116 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:11.761 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:11 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:11.761 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2943487609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-05T07:35:11.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:11 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:11.762 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2943487609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 -- 172.21.15.43:0/3448185374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901029b0 msgr2=0x7f7d90106980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 --2- 172.21.15.43:0/3448185374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901029b0 0x7f7d90106980 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009990 tx=0x7f7d80009ca0 comp rx=0 tx=0).stop 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 -- 172.21.15.43:0/3448185374 shutdown_connections 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 --2- 172.21.15.43:0/3448185374 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d901070b0 0x7f7d9010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 --2- 172.21.15.43:0/3448185374 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d90102000 0x7f7d901023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 --2- 172.21.15.43:0/3448185374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901029b0 0x7f7d90106980 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 -- 172.21.15.43:0/3448185374 >> 172.21.15.43:0/3448185374 conn(0x7f7d90078e40 msgr2=0x7f7d90079250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 -- 172.21.15.43:0/3448185374 shutdown_connections 2024-02-05T07:35:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 -- 172.21.15.43:0/3448185374 wait complete. 2024-02-05T07:35:11.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.879+0000 7f7d96089700 1 Processor -- start 2024-02-05T07:35:11.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d96089700 1 -- start start 2024-02-05T07:35:11.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d96089700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d90102000 0x7f7d9019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:11.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d96089700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d901029b0 0x7f7d9019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d96089700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901070b0 0x7f7d901975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d96089700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90197c50 con 0x7f7d901070b0 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d96089700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90197d90 con 0x7f7d90102000 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d96089700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90197f50 con 0x7f7d901029b0 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d901029b0 0x7f7d9019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d901029b0 0x7f7d9019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:45430/0 (socket says 172.21.15.43:45430) 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 -- 172.21.15.43:0/4146783720 learned_addr learned my addr 172.21.15.43:0/4146783720 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8ffff700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901070b0 0x7f7d901975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8f7fe700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d90102000 0x7f7d9019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 -- 172.21.15.43:0/4146783720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d90102000 msgr2=0x7f7d9019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d90102000 0x7f7d9019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 -- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901070b0 msgr2=0x7f7d901975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901070b0 0x7f7d901975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:11.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.880+0000 7f7d8effd700 1 -- 172.21.15.43:0/4146783720 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78009520 con 0x7f7d901029b0 2024-02-05T07:35:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.881+0000 7f7d8effd700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d901029b0 0x7f7d9019d970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d80000c00 tx=0x7f7d80005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.881+0000 7f7d8cff9700 1 -- 172.21.15.43:0/4146783720 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8001d070 con 0x7f7d901029b0 2024-02-05T07:35:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.881+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d800095f0 con 0x7f7d901029b0 2024-02-05T07:35:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.881+0000 7f7d8cff9700 1 -- 172.21.15.43:0/4146783720 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7d8000bca0 con 0x7f7d901029b0 2024-02-05T07:35:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.881+0000 7f7d8cff9700 1 -- 172.21.15.43:0/4146783720 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d80022c00 con 0x7f7d901029b0 2024-02-05T07:35:11.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.881+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d901a4300 con 0x7f7d901029b0 2024-02-05T07:35:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.882+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d90103460 con 0x7f7d901029b0 2024-02-05T07:35:11.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.882+0000 7f7d8cff9700 1 -- 172.21.15.43:0/4146783720 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f7d8000f460 con 0x7f7d901029b0 2024-02-05T07:35:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.883+0000 7f7d8cff9700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f7d7c06dac0 0x7f7d7c06ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.883+0000 7f7d8cff9700 1 -- 172.21.15.43:0/4146783720 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f7d8008eea0 con 0x7f7d901029b0 2024-02-05T07:35:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.883+0000 7f7d8f7fe700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f7d7c06dac0 0x7f7d7c06ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:11.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.883+0000 7f7d8f7fe700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f7d7c06dac0 0x7f7d7c06ff80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009d80 tx=0x7f7d78009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:11.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:11.888+0000 7f7d8cff9700 1 -- 172.21.15.43:0/4146783720 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d80058320 con 0x7f7d901029b0 2024-02-05T07:35:12.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:11 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:12.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2943487609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-05T07:35:12.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.094+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7d90198b20 con 0x7f7d901029b0 2024-02-05T07:35:12.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.094+0000 7f7d8cff9700 1 -- 172.21.15.43:0/4146783720 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d80027020 con 0x7f7d901029b0 2024-02-05T07:35:12.096 INFO:teuthology.orchestra.run.smithi043.stdout:137438953508 2024-02-05T07:35:12.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f7d7c06dac0 msgr2=0x7f7d7c06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f7d7c06dac0 0x7f7d7c06ff80 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009d80 tx=0x7f7d78009260 comp rx=0 tx=0).stop 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d901029b0 msgr2=0x7f7d9019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d901029b0 0x7f7d9019d970 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7d80000c00 tx=0x7f7d80005740 comp rx=0 tx=0).stop 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 shutdown_connections 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7d901070b0 0x7f7d901975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f7d90102000 0x7f7d9019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f7d7c06dac0 0x7f7d7c06ff80 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 --2- 172.21.15.43:0/4146783720 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f7d901029b0 0x7f7d9019d970 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.096+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 >> 172.21.15.43:0/4146783720 conn(0x7f7d90078e40 msgr2=0x7f7d900ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.097+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 shutdown_connections 2024-02-05T07:35:12.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:12.097+0000 7f7d96089700 1 -- 172.21.15.43:0/4146783720 wait complete. 2024-02-05T07:35:12.366 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149731 got 111669149735 for osd.3 2024-02-05T07:35:12.366 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:13.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:12 smithi043 ceph-mon[29879]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:13.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4146783720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-05T07:35:13.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:12 smithi043 ceph-mon[41444]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:13.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4146783720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-05T07:35:13.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:12 smithi050 ceph-mon[33492]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:13.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4146783720' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-05T07:35:13.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168591 got 240518168594 for osd.7 2024-02-05T07:35:13.214 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:14.214 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953504 got 137438953508 for osd.4 2024-02-05T07:35:14.214 DEBUG:teuthology.parallel:result is None 2024-02-05T07:35:14.214 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-05T07:35:14.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-05T07:35:14.458 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:35:15.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:14 smithi043 ceph-mon[29879]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:15.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:14 smithi043 ceph-mon[41444]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:15.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:14 smithi050 ceph-mon[33492]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:17.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:16 smithi043 ceph-mon[29879]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:16 smithi043 ceph-mon[41444]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:16 smithi050 ceph-mon[33492]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:19.093 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:18 smithi043 ceph-mon[29879]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:19.093 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:18 smithi043 ceph-mon[41444]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 -- 172.21.15.43:0/3007151378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924101860 msgr2=0x7fd924105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 --2- 172.21.15.43:0/3007151378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924101860 0x7fd924105820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd9180098f0 tx=0x7fd918009c00 comp rx=0 tx=0).stop 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 -- 172.21.15.43:0/3007151378 shutdown_connections 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 --2- 172.21.15.43:0/3007151378 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd924105f00 0x7fd92410dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 --2- 172.21.15.43:0/3007151378 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd924100eb0 0x7fd924101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 --2- 172.21.15.43:0/3007151378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924101860 0x7fd924105820 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 -- 172.21.15.43:0/3007151378 >> 172.21.15.43:0/3007151378 conn(0x7fd924076c70 msgr2=0x7fd924077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:19.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 -- 172.21.15.43:0/3007151378 shutdown_connections 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.309+0000 7fd92bfec700 1 -- 172.21.15.43:0/3007151378 wait complete. 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 Processor -- start 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 -- start start 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924100eb0 0x7fd9241028c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd924101860 0x7fd924102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd924105f00 0x7fd924103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a5c70 con 0x7fd924100eb0 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a5de0 con 0x7fd924101860 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92bfec700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9241a5fa0 con 0x7fd924105f00 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924100eb0 0x7fd9241028c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92a589700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd924105f00 0x7fd924103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924100eb0 0x7fd9241028c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52106/0 (socket says 172.21.15.43:52106) 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 -- 172.21.15.43:0/2758343204 learned_addr learned my addr 172.21.15.43:0/2758343204 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929587700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd924101860 0x7fd924102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:19.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 -- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd924105f00 msgr2=0x7fd924103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd924105f00 0x7fd924103340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 -- 172.21.15.43:0/2758343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd924101860 msgr2=0x7fd924102e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd924101860 0x7fd924102e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd929d88700 1 -- 172.21.15.43:0/2758343204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c009d00 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.310+0000 7fd92a589700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd924105f00 0x7fd924103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.311+0000 7fd929d88700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924100eb0 0x7fd9241028c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd92000d950 tx=0x7fd92000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.311+0000 7fd916ffd700 1 -- 172.21.15.43:0/2758343204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000f890 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.311+0000 7fd916ffd700 1 -- 172.21.15.43:0/2758343204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd92000fe70 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.311+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd918009520 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.311+0000 7fd916ffd700 1 -- 172.21.15.43:0/2758343204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92000e3b0 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.311+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9241a65c0 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.312+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9240689e0 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.312+0000 7fd916ffd700 1 -- 172.21.15.43:0/2758343204 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fd92000f9f0 con 0x7fd924100eb0 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.313+0000 7fd916ffd700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd91006dac0 0x7fd91006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:19.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.313+0000 7fd916ffd700 1 -- 172.21.15.43:0/2758343204 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fd92008d650 con 0x7fd924100eb0 2024-02-05T07:35:19.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.313+0000 7fd929587700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd91006dac0 0x7fd91006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:19.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.313+0000 7fd929587700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd91006dac0 0x7fd91006ff80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd918009810 tx=0x7fd918019040 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:19.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.318+0000 7fd916ffd700 1 -- 172.21.15.43:0/2758343204 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd920056a60 con 0x7fd924100eb0 2024-02-05T07:35:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:18 smithi050 ceph-mon[33492]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:19.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.517+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 --> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd924110180 con 0x7fd91006dac0 2024-02-05T07:35:19.519 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:35:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:35:19.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.529+0000 7fd916ffd700 1 -- 172.21.15.43:0/2758343204 <== mgr.24485 v2:172.21.15.43:6800/847956893 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384004 (secure 0 0 0) 0x7fd924110180 con 0x7fd91006dac0 2024-02-05T07:35:19.535 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:35:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd91006dac0 msgr2=0x7fd91006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd91006dac0 0x7fd91006ff80 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd918009810 tx=0x7fd918019040 comp rx=0 tx=0).stop 2024-02-05T07:35:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924100eb0 msgr2=0x7fd9241028c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924100eb0 0x7fd9241028c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd92000d950 tx=0x7fd92000dc60 comp rx=0 tx=0).stop 2024-02-05T07:35:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 shutdown_connections 2024-02-05T07:35:19.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fd91006dac0 0x7fd91006ff80 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fd924105f00 0x7fd924103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd924100eb0 0x7fd9241028c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 --2- 172.21.15.43:0/2758343204 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fd924101860 0x7fd924102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 >> 172.21.15.43:0/2758343204 conn(0x7fd924076c70 msgr2=0x7fd92406a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 shutdown_connections 2024-02-05T07:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:19.534+0000 7fd92bfec700 1 -- 172.21.15.43:0/2758343204 wait complete. 2024-02-05T07:35:19.539 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-02-05T07:35:20.951 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:35:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:35:21.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:20 smithi043 ceph-mon[41444]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:35:21.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:20 smithi043 ceph-mon[41444]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:21.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:20 smithi043 ceph-mon[29879]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:35:21.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:20 smithi043 ceph-mon[29879]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:21.254 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-02-05T07:35:17.636557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1004,"num_read_kb":850,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":245168,"kb_used_data":8220,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716048,"statfs":{"total":767960285184,"available":767709233152,"internally_reserved":0,"allocated":8417280,"data_stored":4534164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003462"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891390+0000","last_change":"2024-02-05T07:34:04.031923+0000","last_active":"2024-02-05T07:34:39.891390+0000","last_peered":"2024-02-05T07:34:39.891390+0000","last_clean":"2024-02-05T07:34:39.891390+0000","last_became_active":"2024-02-05T07:34:04.031632+0000","last_became_peered":"2024-02-05T07:34:04.031632+0000","last_unstale":"2024-02-05T07:34:39.891390+0000","last_undegraded":"2024-02-05T07:34:39.891390+0000","last_fullsized":"2024-02-05T07:34:39.891390+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:25:08.459570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619167+0000","last_change":"2024-02-05T07:34:02.027064+0000","last_active":"2024-02-05T07:34:39.619167+0000","last_peered":"2024-02-05T07:34:39.619167+0000","last_clean":"2024-02-05T07:34:39.619167+0000","last_became_active":"2024-02-05T07:34:02.026763+0000","last_became_peered":"2024-02-05T07:34:02.026763+0000","last_unstale":"2024-02-05T07:34:39.619167+0000","last_undegraded":"2024-02-05T07:34:39.619167+0000","last_fullsized":"2024-02-05T07:34:39.619167+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:35:09.525261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463550+0000","last_change":"2024-02-05T07:33:57.996546+0000","last_active":"2024-02-05T07:34:40.463550+0000","last_peered":"2024-02-05T07:34:40.463550+0000","last_clean":"2024-02-05T07:34:40.463550+0000","last_became_active":"2024-02-05T07:33:57.996078+0000","last_became_peered":"2024-02-05T07:33:57.996078+0000","last_unstale":"2024-02-05T07:34:40.463550+0000","last_undegraded":"2024-02-05T07:34:40.463550+0000","last_fullsized":"2024-02-05T07:34:40.463550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:33:38.836788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891310+0000","last_change":"2024-02-05T07:34:00.014624+0000","last_active":"2024-02-05T07:34:39.891310+0000","last_peered":"2024-02-05T07:34:39.891310+0000","last_clean":"2024-02-05T07:34:39.891310+0000","last_became_active":"2024-02-05T07:34:00.014244+0000","last_became_peered":"2024-02-05T07:34:00.014244+0000","last_unstale":"2024-02-05T07:34:39.891310+0000","last_undegraded":"2024-02-05T07:34:39.891310+0000","last_fullsized":"2024-02-05T07:34:39.891310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:15:10.317337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.872076+0000","last_change":"2024-02-05T07:34:04.369467+0000","last_active":"2024-02-05T07:34:39.872076+0000","last_peered":"2024-02-05T07:34:39.872076+0000","last_clean":"2024-02-05T07:34:39.872076+0000","last_became_active":"2024-02-05T07:34:04.369121+0000","last_became_peered":"2024-02-05T07:34:04.369121+0000","last_unstale":"2024-02-05T07:34:39.872076+0000","last_undegraded":"2024-02-05T07:34:39.872076+0000","last_fullsized":"2024-02-05T07:34:39.872076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:36:08.873593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891313+0000","last_change":"2024-02-05T07:34:02.024818+0000","last_active":"2024-02-05T07:34:39.891313+0000","last_peered":"2024-02-05T07:34:39.891313+0000","last_clean":"2024-02-05T07:34:39.891313+0000","last_became_active":"2024-02-05T07:34:02.024497+0000","last_became_peered":"2024-02-05T07:34:02.024497+0000","last_unstale":"2024-02-05T07:34:39.891313+0000","last_undegraded":"2024-02-05T07:34:39.891313+0000","last_fullsized":"2024-02-05T07:34:39.891313+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:25:48.725488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871897+0000","last_change":"2024-02-05T07:33:57.996910+0000","last_active":"2024-02-05T07:34:39.871897+0000","last_peered":"2024-02-05T07:34:39.871897+0000","last_clean":"2024-02-05T07:34:39.871897+0000","last_became_active":"2024-02-05T07:33:57.996609+0000","last_became_peered":"2024-02-05T07:33:57.996609+0000","last_unstale":"2024-02-05T07:34:39.871897+0000","last_undegraded":"2024-02-05T07:34:39.871897+0000","last_fullsized":"2024-02-05T07:34:39.871897+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:57:51.437191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871369+0000","last_change":"2024-02-05T07:34:00.018001+0000","last_active":"2024-02-05T07:34:39.871369+0000","last_peered":"2024-02-05T07:34:39.871369+0000","last_clean":"2024-02-05T07:34:39.871369+0000","last_became_active":"2024-02-05T07:34:00.017656+0000","last_became_peered":"2024-02-05T07:34:00.017656+0000","last_unstale":"2024-02-05T07:34:39.871369+0000","last_undegraded":"2024-02-05T07:34:39.871369+0000","last_fullsized":"2024-02-05T07:34:39.871369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:01:53.164886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462878+0000","last_change":"2024-02-05T07:34:04.042609+0000","last_active":"2024-02-05T07:34:40.462878+0000","last_peered":"2024-02-05T07:34:40.462878+0000","last_clean":"2024-02-05T07:34:40.462878+0000","last_became_active":"2024-02-05T07:34:04.042264+0000","last_became_peered":"2024-02-05T07:34:04.042264+0000","last_unstale":"2024-02-05T07:34:40.462878+0000","last_undegraded":"2024-02-05T07:34:40.462878+0000","last_fullsized":"2024-02-05T07:34:40.462878+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:51:41.134652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723801+0000","last_change":"2024-02-05T07:34:02.025982+0000","last_active":"2024-02-05T07:34:39.723801+0000","last_peered":"2024-02-05T07:34:39.723801+0000","last_clean":"2024-02-05T07:34:39.723801+0000","last_became_active":"2024-02-05T07:34:02.025683+0000","last_became_peered":"2024-02-05T07:34:02.025683+0000","last_unstale":"2024-02-05T07:34:39.723801+0000","last_undegraded":"2024-02-05T07:34:39.723801+0000","last_fullsized":"2024-02-05T07:34:39.723801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:31:36.871597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724093+0000","last_change":"2024-02-05T07:33:57.998743+0000","last_active":"2024-02-05T07:34:39.724093+0000","last_peered":"2024-02-05T07:34:39.724093+0000","last_clean":"2024-02-05T07:34:39.724093+0000","last_became_active":"2024-02-05T07:33:57.998414+0000","last_became_peered":"2024-02-05T07:33:57.998414+0000","last_unstale":"2024-02-05T07:34:39.724093+0000","last_undegraded":"2024-02-05T07:34:39.724093+0000","last_fullsized":"2024-02-05T07:34:39.724093+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:29:42.546996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891585+0000","last_change":"2024-02-05T07:34:00.012151+0000","last_active":"2024-02-05T07:34:39.891585+0000","last_peered":"2024-02-05T07:34:39.891585+0000","last_clean":"2024-02-05T07:34:39.891585+0000","last_became_active":"2024-02-05T07:34:00.011747+0000","last_became_peered":"2024-02-05T07:34:00.011747+0000","last_unstale":"2024-02-05T07:34:39.891585+0000","last_undegraded":"2024-02-05T07:34:39.891585+0000","last_fullsized":"2024-02-05T07:34:39.891585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:25:46.642968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723514+0000","last_change":"2024-02-05T07:34:04.039520+0000","last_active":"2024-02-05T07:34:39.723514+0000","last_peered":"2024-02-05T07:34:39.723514+0000","last_clean":"2024-02-05T07:34:39.723514+0000","last_became_active":"2024-02-05T07:34:04.039182+0000","last_became_peered":"2024-02-05T07:34:04.039182+0000","last_unstale":"2024-02-05T07:34:39.723514+0000","last_undegraded":"2024-02-05T07:34:39.723514+0000","last_fullsized":"2024-02-05T07:34:39.723514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:04:39.438940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734773+0000","last_change":"2024-02-05T07:34:02.029675+0000","last_active":"2024-02-05T07:34:39.734773+0000","last_peered":"2024-02-05T07:34:39.734773+0000","last_clean":"2024-02-05T07:34:39.734773+0000","last_became_active":"2024-02-05T07:34:02.029447+0000","last_became_peered":"2024-02-05T07:34:02.029447+0000","last_unstale":"2024-02-05T07:34:39.734773+0000","last_undegraded":"2024-02-05T07:34:39.734773+0000","last_fullsized":"2024-02-05T07:34:39.734773+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:37:24.173720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723938+0000","last_change":"2024-02-05T07:33:58.000439+0000","last_active":"2024-02-05T07:34:39.723938+0000","last_peered":"2024-02-05T07:34:39.723938+0000","last_clean":"2024-02-05T07:34:39.723938+0000","last_became_active":"2024-02-05T07:33:57.999986+0000","last_became_peered":"2024-02-05T07:33:57.999986+0000","last_unstale":"2024-02-05T07:34:39.723938+0000","last_undegraded":"2024-02-05T07:34:39.723938+0000","last_fullsized":"2024-02-05T07:34:39.723938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:59:51.789954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891825+0000","last_change":"2024-02-05T07:34:00.015319+0000","last_active":"2024-02-05T07:34:39.891825+0000","last_peered":"2024-02-05T07:34:39.891825+0000","last_clean":"2024-02-05T07:34:39.891825+0000","last_became_active":"2024-02-05T07:34:00.014964+0000","last_became_peered":"2024-02-05T07:34:00.014964+0000","last_unstale":"2024-02-05T07:34:39.891825+0000","last_undegraded":"2024-02-05T07:34:39.891825+0000","last_fullsized":"2024-02-05T07:34:39.891825+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:54:35.412027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892217+0000","last_change":"2024-02-05T07:34:04.368376+0000","last_active":"2024-02-05T07:34:39.892217+0000","last_peered":"2024-02-05T07:34:39.892217+0000","last_clean":"2024-02-05T07:34:39.892217+0000","last_became_active":"2024-02-05T07:34:04.367455+0000","last_became_peered":"2024-02-05T07:34:04.367455+0000","last_unstale":"2024-02-05T07:34:39.892217+0000","last_undegraded":"2024-02-05T07:34:39.892217+0000","last_fullsized":"2024-02-05T07:34:39.892217+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:41:02.971468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619675+0000","last_change":"2024-02-05T07:34:02.024447+0000","last_active":"2024-02-05T07:34:39.619675+0000","last_peered":"2024-02-05T07:34:39.619675+0000","last_clean":"2024-02-05T07:34:39.619675+0000","last_became_active":"2024-02-05T07:34:02.024123+0000","last_became_peered":"2024-02-05T07:34:02.024123+0000","last_unstale":"2024-02-05T07:34:39.619675+0000","last_undegraded":"2024-02-05T07:34:39.619675+0000","last_fullsized":"2024-02-05T07:34:39.619675+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:57:57.972408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463231+0000","last_change":"2024-02-05T07:33:58.001920+0000","last_active":"2024-02-05T07:34:40.463231+0000","last_peered":"2024-02-05T07:34:40.463231+0000","last_clean":"2024-02-05T07:34:40.463231+0000","last_became_active":"2024-02-05T07:33:58.001592+0000","last_became_peered":"2024-02-05T07:33:58.001592+0000","last_unstale":"2024-02-05T07:34:40.463231+0000","last_undegraded":"2024-02-05T07:34:40.463231+0000","last_fullsized":"2024-02-05T07:34:40.463231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:33:33.448769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617693+0000","last_change":"2024-02-05T07:34:00.016711+0000","last_active":"2024-02-05T07:34:39.617693+0000","last_peered":"2024-02-05T07:34:39.617693+0000","last_clean":"2024-02-05T07:34:39.617693+0000","last_became_active":"2024-02-05T07:34:00.016363+0000","last_became_peered":"2024-02-05T07:34:00.016363+0000","last_unstale":"2024-02-05T07:34:39.617693+0000","last_undegraded":"2024-02-05T07:34:39.617693+0000","last_fullsized":"2024-02-05T07:34:39.617693+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:44:11.434929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871568+0000","last_change":"2024-02-05T07:34:04.368183+0000","last_active":"2024-02-05T07:34:39.871568+0000","last_peered":"2024-02-05T07:34:39.871568+0000","last_clean":"2024-02-05T07:34:39.871568+0000","last_became_active":"2024-02-05T07:34:04.367787+0000","last_became_peered":"2024-02-05T07:34:04.367787+0000","last_unstale":"2024-02-05T07:34:39.871568+0000","last_undegraded":"2024-02-05T07:34:39.871568+0000","last_fullsized":"2024-02-05T07:34:39.871568+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:13:56.195144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891661+0000","last_change":"2024-02-05T07:34:02.027658+0000","last_active":"2024-02-05T07:34:39.891661+0000","last_peered":"2024-02-05T07:34:39.891661+0000","last_clean":"2024-02-05T07:34:39.891661+0000","last_became_active":"2024-02-05T07:34:02.027365+0000","last_became_peered":"2024-02-05T07:34:02.027365+0000","last_unstale":"2024-02-05T07:34:39.891661+0000","last_undegraded":"2024-02-05T07:34:39.891661+0000","last_fullsized":"2024-02-05T07:34:39.891661+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:53:47.907255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891621+0000","last_change":"2024-02-05T07:33:58.001194+0000","last_active":"2024-02-05T07:34:39.891621+0000","last_peered":"2024-02-05T07:34:39.891621+0000","last_clean":"2024-02-05T07:34:39.891621+0000","last_became_active":"2024-02-05T07:33:58.000798+0000","last_became_peered":"2024-02-05T07:33:58.000798+0000","last_unstale":"2024-02-05T07:34:39.891621+0000","last_undegraded":"2024-02-05T07:34:39.891621+0000","last_fullsized":"2024-02-05T07:34:39.891621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:41:33.103435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871781+0000","last_change":"2024-02-05T07:34:00.017876+0000","last_active":"2024-02-05T07:34:39.871781+0000","last_peered":"2024-02-05T07:34:39.871781+0000","last_clean":"2024-02-05T07:34:39.871781+0000","last_became_active":"2024-02-05T07:34:00.017471+0000","last_became_peered":"2024-02-05T07:34:00.017471+0000","last_unstale":"2024-02-05T07:34:39.871781+0000","last_undegraded":"2024-02-05T07:34:39.871781+0000","last_fullsized":"2024-02-05T07:34:39.871781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:13:28.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735186+0000","last_change":"2024-02-05T07:34:04.035628+0000","last_active":"2024-02-05T07:34:39.735186+0000","last_peered":"2024-02-05T07:34:39.735186+0000","last_clean":"2024-02-05T07:34:39.735186+0000","last_became_active":"2024-02-05T07:34:04.035317+0000","last_became_peered":"2024-02-05T07:34:04.035317+0000","last_unstale":"2024-02-05T07:34:39.735186+0000","last_undegraded":"2024-02-05T07:34:39.735186+0000","last_fullsized":"2024-02-05T07:34:39.735186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:14:58.016909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059663+0000","last_change":"2024-02-05T07:34:02.023399+0000","last_active":"2024-02-05T07:35:08.059663+0000","last_peered":"2024-02-05T07:35:08.059663+0000","last_clean":"2024-02-05T07:35:08.059663+0000","last_became_active":"2024-02-05T07:34:02.023100+0000","last_became_peered":"2024-02-05T07:34:02.023100+0000","last_unstale":"2024-02-05T07:35:08.059663+0000","last_undegraded":"2024-02-05T07:35:08.059663+0000","last_fullsized":"2024-02-05T07:35:08.059663+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:56:16.025653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619268+0000","last_change":"2024-02-05T07:33:58.000793+0000","last_active":"2024-02-05T07:34:39.619268+0000","last_peered":"2024-02-05T07:34:39.619268+0000","last_clean":"2024-02-05T07:34:39.619268+0000","last_became_active":"2024-02-05T07:33:58.000509+0000","last_became_peered":"2024-02-05T07:33:58.000509+0000","last_unstale":"2024-02-05T07:34:39.619268+0000","last_undegraded":"2024-02-05T07:34:39.619268+0000","last_fullsized":"2024-02-05T07:34:39.619268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:22:07.034225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462730+0000","last_change":"2024-02-05T07:34:00.268609+0000","last_active":"2024-02-05T07:34:40.462730+0000","last_peered":"2024-02-05T07:34:40.462730+0000","last_clean":"2024-02-05T07:34:40.462730+0000","last_became_active":"2024-02-05T07:34:00.268317+0000","last_became_peered":"2024-02-05T07:34:00.268317+0000","last_unstale":"2024-02-05T07:34:40.462730+0000","last_undegraded":"2024-02-05T07:34:40.462730+0000","last_fullsized":"2024-02-05T07:34:40.462730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:51:42.501493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619866+0000","last_change":"2024-02-05T07:34:04.040699+0000","last_active":"2024-02-05T07:34:39.619866+0000","last_peered":"2024-02-05T07:34:39.619866+0000","last_clean":"2024-02-05T07:34:39.619866+0000","last_became_active":"2024-02-05T07:34:04.036768+0000","last_became_peered":"2024-02-05T07:34:04.036768+0000","last_unstale":"2024-02-05T07:34:39.619866+0000","last_undegraded":"2024-02-05T07:34:39.619866+0000","last_fullsized":"2024-02-05T07:34:39.619866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:16:50.721705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463557+0000","last_change":"2024-02-05T07:34:02.029568+0000","last_active":"2024-02-05T07:34:40.463557+0000","last_peered":"2024-02-05T07:34:40.463557+0000","last_clean":"2024-02-05T07:34:40.463557+0000","last_became_active":"2024-02-05T07:34:02.028050+0000","last_became_peered":"2024-02-05T07:34:02.028050+0000","last_unstale":"2024-02-05T07:34:40.463557+0000","last_undegraded":"2024-02-05T07:34:40.463557+0000","last_fullsized":"2024-02-05T07:34:40.463557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:48:23.146415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870556+0000","last_change":"2024-02-05T07:33:57.999919+0000","last_active":"2024-02-05T07:34:39.870556+0000","last_peered":"2024-02-05T07:34:39.870556+0000","last_clean":"2024-02-05T07:34:39.870556+0000","last_became_active":"2024-02-05T07:33:57.999622+0000","last_became_peered":"2024-02-05T07:33:57.999622+0000","last_unstale":"2024-02-05T07:34:39.870556+0000","last_undegraded":"2024-02-05T07:34:39.870556+0000","last_fullsized":"2024-02-05T07:34:39.870556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:55:35.622235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.618911+0000","last_change":"2024-02-05T07:34:00.017655+0000","last_active":"2024-02-05T07:34:39.618911+0000","last_peered":"2024-02-05T07:34:39.618911+0000","last_clean":"2024-02-05T07:34:39.618911+0000","last_became_active":"2024-02-05T07:34:00.017281+0000","last_became_peered":"2024-02-05T07:34:00.017281+0000","last_unstale":"2024-02-05T07:34:39.618911+0000","last_undegraded":"2024-02-05T07:34:39.618911+0000","last_fullsized":"2024-02-05T07:34:39.618911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:39:04.393070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734765+0000","last_change":"2024-02-05T07:34:04.035415+0000","last_active":"2024-02-05T07:34:39.734765+0000","last_peered":"2024-02-05T07:34:39.734765+0000","last_clean":"2024-02-05T07:34:39.734765+0000","last_became_active":"2024-02-05T07:34:04.035027+0000","last_became_peered":"2024-02-05T07:34:04.035027+0000","last_unstale":"2024-02-05T07:34:39.734765+0000","last_undegraded":"2024-02-05T07:34:39.734765+0000","last_fullsized":"2024-02-05T07:34:39.734765+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:11:05.242662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.618255+0000","last_change":"2024-02-05T07:34:02.028707+0000","last_active":"2024-02-05T07:34:39.618255+0000","last_peered":"2024-02-05T07:34:39.618255+0000","last_clean":"2024-02-05T07:34:39.618255+0000","last_became_active":"2024-02-05T07:34:02.028012+0000","last_became_peered":"2024-02-05T07:34:02.028012+0000","last_unstale":"2024-02-05T07:34:39.618255+0000","last_undegraded":"2024-02-05T07:34:39.618255+0000","last_fullsized":"2024-02-05T07:34:39.618255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:23:57.296432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463515+0000","last_change":"2024-02-05T07:33:58.006055+0000","last_active":"2024-02-05T07:34:40.463515+0000","last_peered":"2024-02-05T07:34:40.463515+0000","last_clean":"2024-02-05T07:34:40.463515+0000","last_became_active":"2024-02-05T07:33:58.004979+0000","last_became_peered":"2024-02-05T07:33:58.004979+0000","last_unstale":"2024-02-05T07:34:40.463515+0000","last_undegraded":"2024-02-05T07:34:40.463515+0000","last_fullsized":"2024-02-05T07:34:40.463515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:47:46.304781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870628+0000","last_change":"2024-02-05T07:34:00.017835+0000","last_active":"2024-02-05T07:34:39.870628+0000","last_peered":"2024-02-05T07:34:39.870628+0000","last_clean":"2024-02-05T07:34:39.870628+0000","last_became_active":"2024-02-05T07:34:00.017151+0000","last_became_peered":"2024-02-05T07:34:00.017151+0000","last_unstale":"2024-02-05T07:34:39.870628+0000","last_undegraded":"2024-02-05T07:34:39.870628+0000","last_fullsized":"2024-02-05T07:34:39.870628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:35:31.486048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870933+0000","last_change":"2024-02-05T07:34:04.367657+0000","last_active":"2024-02-05T07:34:39.870933+0000","last_peered":"2024-02-05T07:34:39.870933+0000","last_clean":"2024-02-05T07:34:39.870933+0000","last_became_active":"2024-02-05T07:34:04.367251+0000","last_became_peered":"2024-02-05T07:34:04.367251+0000","last_unstale":"2024-02-05T07:34:39.870933+0000","last_undegraded":"2024-02-05T07:34:39.870933+0000","last_fullsized":"2024-02-05T07:34:39.870933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:32:24.063406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734985+0000","last_change":"2024-02-05T07:34:02.030849+0000","last_active":"2024-02-05T07:34:39.734985+0000","last_peered":"2024-02-05T07:34:39.734985+0000","last_clean":"2024-02-05T07:34:39.734985+0000","last_became_active":"2024-02-05T07:34:02.030572+0000","last_became_peered":"2024-02-05T07:34:02.030572+0000","last_unstale":"2024-02-05T07:34:39.734985+0000","last_undegraded":"2024-02-05T07:34:39.734985+0000","last_fullsized":"2024-02-05T07:34:39.734985+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:54:30.418129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724202+0000","last_change":"2024-02-05T07:33:58.001589+0000","last_active":"2024-02-05T07:34:39.724202+0000","last_peered":"2024-02-05T07:34:39.724202+0000","last_clean":"2024-02-05T07:34:39.724202+0000","last_became_active":"2024-02-05T07:33:58.001239+0000","last_became_peered":"2024-02-05T07:33:58.001239+0000","last_unstale":"2024-02-05T07:34:39.724202+0000","last_undegraded":"2024-02-05T07:34:39.724202+0000","last_fullsized":"2024-02-05T07:34:39.724202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:00:58.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.872013+0000","last_change":"2024-02-05T07:34:00.267061+0000","last_active":"2024-02-05T07:34:39.872013+0000","last_peered":"2024-02-05T07:34:39.872013+0000","last_clean":"2024-02-05T07:34:39.872013+0000","last_became_active":"2024-02-05T07:34:00.266778+0000","last_became_peered":"2024-02-05T07:34:00.266778+0000","last_unstale":"2024-02-05T07:34:39.872013+0000","last_undegraded":"2024-02-05T07:34:39.872013+0000","last_fullsized":"2024-02-05T07:34:39.872013+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:44:46.908614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462618+0000","last_change":"2024-02-05T07:34:04.041064+0000","last_active":"2024-02-05T07:34:40.462618+0000","last_peered":"2024-02-05T07:34:40.462618+0000","last_clean":"2024-02-05T07:34:40.462618+0000","last_became_active":"2024-02-05T07:34:04.040740+0000","last_became_peered":"2024-02-05T07:34:04.040740+0000","last_unstale":"2024-02-05T07:34:40.462618+0000","last_undegraded":"2024-02-05T07:34:40.462618+0000","last_fullsized":"2024-02-05T07:34:40.462618+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:42:54.475455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871664+0000","last_change":"2024-02-05T07:34:02.024868+0000","last_active":"2024-02-05T07:34:39.871664+0000","last_peered":"2024-02-05T07:34:39.871664+0000","last_clean":"2024-02-05T07:34:39.871664+0000","last_became_active":"2024-02-05T07:34:02.024582+0000","last_became_peered":"2024-02-05T07:34:02.024582+0000","last_unstale":"2024-02-05T07:34:39.871664+0000","last_undegraded":"2024-02-05T07:34:39.871664+0000","last_fullsized":"2024-02-05T07:34:39.871664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:22:11.932379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735275+0000","last_change":"2024-02-05T07:33:58.002496+0000","last_active":"2024-02-05T07:34:39.735275+0000","last_peered":"2024-02-05T07:34:39.735275+0000","last_clean":"2024-02-05T07:34:39.735275+0000","last_became_active":"2024-02-05T07:33:58.002039+0000","last_became_peered":"2024-02-05T07:33:58.002039+0000","last_unstale":"2024-02-05T07:34:39.735275+0000","last_undegraded":"2024-02-05T07:34:39.735275+0000","last_fullsized":"2024-02-05T07:34:39.735275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:11:20.328274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723568+0000","last_change":"2024-02-05T07:34:16.962178+0000","last_active":"2024-02-05T07:34:39.723568+0000","last_peered":"2024-02-05T07:34:39.723568+0000","last_clean":"2024-02-05T07:34:39.723568+0000","last_became_active":"2024-02-05T07:34:16.954846+0000","last_became_peered":"2024-02-05T07:34:16.954846+0000","last_unstale":"2024-02-05T07:34:39.723568+0000","last_undegraded":"2024-02-05T07:34:39.723568+0000","last_fullsized":"2024-02-05T07:34:39.723568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:33:16.266150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870844+0000","last_change":"2024-02-05T07:34:04.037934+0000","last_active":"2024-02-05T07:34:39.870844+0000","last_peered":"2024-02-05T07:34:39.870844+0000","last_clean":"2024-02-05T07:34:39.870844+0000","last_became_active":"2024-02-05T07:34:04.037562+0000","last_became_peered":"2024-02-05T07:34:04.037562+0000","last_unstale":"2024-02-05T07:34:39.870844+0000","last_undegraded":"2024-02-05T07:34:39.870844+0000","last_fullsized":"2024-02-05T07:34:39.870844+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:45:16.032769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620350+0000","last_change":"2024-02-05T07:34:02.025940+0000","last_active":"2024-02-05T07:34:39.620350+0000","last_peered":"2024-02-05T07:34:39.620350+0000","last_clean":"2024-02-05T07:34:39.620350+0000","last_became_active":"2024-02-05T07:34:02.025602+0000","last_became_peered":"2024-02-05T07:34:02.025602+0000","last_unstale":"2024-02-05T07:34:39.620350+0000","last_undegraded":"2024-02-05T07:34:39.620350+0000","last_fullsized":"2024-02-05T07:34:39.620350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:34:03.050275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891913+0000","last_change":"2024-02-05T07:33:58.004045+0000","last_active":"2024-02-05T07:34:39.891913+0000","last_peered":"2024-02-05T07:34:39.891913+0000","last_clean":"2024-02-05T07:34:39.891913+0000","last_became_active":"2024-02-05T07:33:58.003717+0000","last_became_peered":"2024-02-05T07:33:58.003717+0000","last_unstale":"2024-02-05T07:34:39.891913+0000","last_undegraded":"2024-02-05T07:34:39.891913+0000","last_fullsized":"2024-02-05T07:34:39.891913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:59:10.423222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891696+0000","last_change":"2024-02-05T07:34:00.014998+0000","last_active":"2024-02-05T07:34:39.891696+0000","last_peered":"2024-02-05T07:34:39.891696+0000","last_clean":"2024-02-05T07:34:39.891696+0000","last_became_active":"2024-02-05T07:34:00.014647+0000","last_became_peered":"2024-02-05T07:34:00.014647+0000","last_unstale":"2024-02-05T07:34:39.891696+0000","last_undegraded":"2024-02-05T07:34:39.891696+0000","last_fullsized":"2024-02-05T07:34:39.891696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:47:53.135296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463458+0000","last_change":"2024-02-05T07:34:04.039256+0000","last_active":"2024-02-05T07:34:40.463458+0000","last_peered":"2024-02-05T07:34:40.463458+0000","last_clean":"2024-02-05T07:34:40.463458+0000","last_became_active":"2024-02-05T07:34:04.038858+0000","last_became_peered":"2024-02-05T07:34:04.038858+0000","last_unstale":"2024-02-05T07:34:40.463458+0000","last_undegraded":"2024-02-05T07:34:40.463458+0000","last_fullsized":"2024-02-05T07:34:40.463458+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:14:50.172470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059637+0000","last_change":"2024-02-05T07:34:02.032169+0000","last_active":"2024-02-05T07:35:08.059637+0000","last_peered":"2024-02-05T07:35:08.059637+0000","last_clean":"2024-02-05T07:35:08.059637+0000","last_became_active":"2024-02-05T07:34:02.031873+0000","last_became_peered":"2024-02-05T07:34:02.031873+0000","last_unstale":"2024-02-05T07:35:08.059637+0000","last_undegraded":"2024-02-05T07:35:08.059637+0000","last_fullsized":"2024-02-05T07:35:08.059637+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:25:37.637027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734872+0000","last_change":"2024-02-05T07:33:58.001784+0000","last_active":"2024-02-05T07:34:39.734872+0000","last_peered":"2024-02-05T07:34:39.734872+0000","last_clean":"2024-02-05T07:34:39.734872+0000","last_became_active":"2024-02-05T07:33:58.001426+0000","last_became_peered":"2024-02-05T07:33:58.001426+0000","last_unstale":"2024-02-05T07:34:39.734872+0000","last_undegraded":"2024-02-05T07:34:39.734872+0000","last_fullsized":"2024-02-05T07:34:39.734872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:17:43.462815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871896+0000","last_change":"2024-02-05T07:34:16.968302+0000","last_active":"2024-02-05T07:34:39.871896+0000","last_peered":"2024-02-05T07:34:39.871896+0000","last_clean":"2024-02-05T07:34:39.871896+0000","last_became_active":"2024-02-05T07:34:16.955303+0000","last_became_peered":"2024-02-05T07:34:16.955303+0000","last_unstale":"2024-02-05T07:34:39.871896+0000","last_undegraded":"2024-02-05T07:34:39.871896+0000","last_fullsized":"2024-02-05T07:34:39.871896+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:52:03.552702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892333+0000","last_change":"2024-02-05T07:34:04.038358+0000","last_active":"2024-02-05T07:34:39.892333+0000","last_peered":"2024-02-05T07:34:39.892333+0000","last_clean":"2024-02-05T07:34:39.892333+0000","last_became_active":"2024-02-05T07:34:04.037736+0000","last_became_peered":"2024-02-05T07:34:04.037736+0000","last_unstale":"2024-02-05T07:34:39.892333+0000","last_undegraded":"2024-02-05T07:34:39.892333+0000","last_fullsized":"2024-02-05T07:34:39.892333+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:58:26.728709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059674+0000","last_change":"2024-02-05T07:34:02.024289+0000","last_active":"2024-02-05T07:35:08.059674+0000","last_peered":"2024-02-05T07:35:08.059674+0000","last_clean":"2024-02-05T07:35:08.059674+0000","last_became_active":"2024-02-05T07:34:02.023960+0000","last_became_peered":"2024-02-05T07:34:02.023960+0000","last_unstale":"2024-02-05T07:35:08.059674+0000","last_undegraded":"2024-02-05T07:35:08.059674+0000","last_fullsized":"2024-02-05T07:35:08.059674+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:03:01.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463110+0000","last_change":"2024-02-05T07:33:57.996408+0000","last_active":"2024-02-05T07:34:40.463110+0000","last_peered":"2024-02-05T07:34:40.463110+0000","last_clean":"2024-02-05T07:34:40.463110+0000","last_became_active":"2024-02-05T07:33:57.996091+0000","last_became_peered":"2024-02-05T07:33:57.996091+0000","last_unstale":"2024-02-05T07:34:40.463110+0000","last_undegraded":"2024-02-05T07:34:40.463110+0000","last_fullsized":"2024-02-05T07:34:40.463110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:34:50.649331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724307+0000","last_change":"2024-02-05T07:34:00.268943+0000","last_active":"2024-02-05T07:34:39.724307+0000","last_peered":"2024-02-05T07:34:39.724307+0000","last_clean":"2024-02-05T07:34:39.724307+0000","last_became_active":"2024-02-05T07:34:00.268603+0000","last_became_peered":"2024-02-05T07:34:00.268603+0000","last_unstale":"2024-02-05T07:34:39.724307+0000","last_undegraded":"2024-02-05T07:34:39.724307+0000","last_fullsized":"2024-02-05T07:34:39.724307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:01:20.645935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617859+0000","last_change":"2024-02-05T07:34:04.033809+0000","last_active":"2024-02-05T07:34:39.617859+0000","last_peered":"2024-02-05T07:34:39.617859+0000","last_clean":"2024-02-05T07:34:39.617859+0000","last_became_active":"2024-02-05T07:34:04.033503+0000","last_became_peered":"2024-02-05T07:34:04.033503+0000","last_unstale":"2024-02-05T07:34:39.617859+0000","last_undegraded":"2024-02-05T07:34:39.617859+0000","last_fullsized":"2024-02-05T07:34:39.617859+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:11:59.491170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871477+0000","last_change":"2024-02-05T07:34:02.025617+0000","last_active":"2024-02-05T07:34:39.871477+0000","last_peered":"2024-02-05T07:34:39.871477+0000","last_clean":"2024-02-05T07:34:39.871477+0000","last_became_active":"2024-02-05T07:34:02.025325+0000","last_became_peered":"2024-02-05T07:34:02.025325+0000","last_unstale":"2024-02-05T07:34:39.871477+0000","last_undegraded":"2024-02-05T07:34:39.871477+0000","last_fullsized":"2024-02-05T07:34:39.871477+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:48:15.389657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734755+0000","last_change":"2024-02-05T07:33:57.998300+0000","last_active":"2024-02-05T07:34:39.734755+0000","last_peered":"2024-02-05T07:34:39.734755+0000","last_clean":"2024-02-05T07:34:39.734755+0000","last_became_active":"2024-02-05T07:33:57.998008+0000","last_became_peered":"2024-02-05T07:33:57.998008+0000","last_unstale":"2024-02-05T07:34:39.734755+0000","last_undegraded":"2024-02-05T07:34:39.734755+0000","last_fullsized":"2024-02-05T07:34:39.734755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:41:35.493577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462569+0000","last_change":"2024-02-05T07:34:00.269352+0000","last_active":"2024-02-05T07:34:40.462569+0000","last_peered":"2024-02-05T07:34:40.462569+0000","last_clean":"2024-02-05T07:34:40.462569+0000","last_became_active":"2024-02-05T07:34:00.269046+0000","last_became_peered":"2024-02-05T07:34:00.269046+0000","last_unstale":"2024-02-05T07:34:40.462569+0000","last_undegraded":"2024-02-05T07:34:40.462569+0000","last_fullsized":"2024-02-05T07:34:40.462569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:08:19.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735264+0000","last_change":"2024-02-05T07:34:04.369274+0000","last_active":"2024-02-05T07:34:39.735264+0000","last_peered":"2024-02-05T07:34:39.735264+0000","last_clean":"2024-02-05T07:34:39.735264+0000","last_became_active":"2024-02-05T07:34:04.368916+0000","last_became_peered":"2024-02-05T07:34:04.368916+0000","last_unstale":"2024-02-05T07:34:39.735264+0000","last_undegraded":"2024-02-05T07:34:39.735264+0000","last_fullsized":"2024-02-05T07:34:39.735264+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:56:29.979066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724292+0000","last_change":"2024-02-05T07:34:02.023606+0000","last_active":"2024-02-05T07:34:39.724292+0000","last_peered":"2024-02-05T07:34:39.724292+0000","last_clean":"2024-02-05T07:34:39.724292+0000","last_became_active":"2024-02-05T07:34:02.023313+0000","last_became_peered":"2024-02-05T07:34:02.023313+0000","last_unstale":"2024-02-05T07:34:39.724292+0000","last_undegraded":"2024-02-05T07:34:39.724292+0000","last_fullsized":"2024-02-05T07:34:39.724292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:59:45.136815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.618209+0000","last_change":"2024-02-05T07:33:57.996598+0000","last_active":"2024-02-05T07:34:39.618209+0000","last_peered":"2024-02-05T07:34:39.618209+0000","last_clean":"2024-02-05T07:34:39.618209+0000","last_became_active":"2024-02-05T07:33:57.996279+0000","last_became_peered":"2024-02-05T07:33:57.996279+0000","last_unstale":"2024-02-05T07:34:39.618209+0000","last_undegraded":"2024-02-05T07:34:39.618209+0000","last_fullsized":"2024-02-05T07:34:39.618209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:10:55.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871670+0000","last_change":"2024-02-05T07:34:00.268196+0000","last_active":"2024-02-05T07:34:39.871670+0000","last_peered":"2024-02-05T07:34:39.871670+0000","last_clean":"2024-02-05T07:34:39.871670+0000","last_became_active":"2024-02-05T07:34:00.267853+0000","last_became_peered":"2024-02-05T07:34:00.267853+0000","last_unstale":"2024-02-05T07:34:39.871670+0000","last_undegraded":"2024-02-05T07:34:39.871670+0000","last_fullsized":"2024-02-05T07:34:39.871670+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:28:54.227264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723845+0000","last_change":"2024-02-05T07:34:04.367690+0000","last_active":"2024-02-05T07:34:39.723845+0000","last_peered":"2024-02-05T07:34:39.723845+0000","last_clean":"2024-02-05T07:34:39.723845+0000","last_became_active":"2024-02-05T07:34:04.367306+0000","last_became_peered":"2024-02-05T07:34:04.367306+0000","last_unstale":"2024-02-05T07:34:39.723845+0000","last_undegraded":"2024-02-05T07:34:39.723845+0000","last_fullsized":"2024-02-05T07:34:39.723845+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:55:11.452215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'3","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735555+0000","last_change":"2024-02-05T07:33:58.004760+0000","last_active":"2024-02-05T07:34:39.735555+0000","last_peered":"2024-02-05T07:34:39.735555+0000","last_clean":"2024-02-05T07:34:39.735555+0000","last_became_active":"2024-02-05T07:33:58.003077+0000","last_became_peered":"2024-02-05T07:33:58.003077+0000","last_unstale":"2024-02-05T07:34:39.735555+0000","last_undegraded":"2024-02-05T07:34:39.735555+0000","last_fullsized":"2024-02-05T07:34:39.735555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:41:41.829590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723572+0000","last_change":"2024-02-05T07:34:00.269682+0000","last_active":"2024-02-05T07:34:39.723572+0000","last_peered":"2024-02-05T07:34:39.723572+0000","last_clean":"2024-02-05T07:34:39.723572+0000","last_became_active":"2024-02-05T07:34:00.269318+0000","last_became_peered":"2024-02-05T07:34:00.269318+0000","last_unstale":"2024-02-05T07:34:39.723572+0000","last_undegraded":"2024-02-05T07:34:39.723572+0000","last_fullsized":"2024-02-05T07:34:39.723572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:34:01.826854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059859+0000","last_change":"2024-02-05T07:34:02.026069+0000","last_active":"2024-02-05T07:35:08.059859+0000","last_peered":"2024-02-05T07:35:08.059859+0000","last_clean":"2024-02-05T07:35:08.059859+0000","last_became_active":"2024-02-05T07:34:02.025793+0000","last_became_peered":"2024-02-05T07:34:02.025793+0000","last_unstale":"2024-02-05T07:35:08.059859+0000","last_undegraded":"2024-02-05T07:35:08.059859+0000","last_fullsized":"2024-02-05T07:35:08.059859+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:49:25.811278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871575+0000","last_change":"2024-02-05T07:34:04.040764+0000","last_active":"2024-02-05T07:34:39.871575+0000","last_peered":"2024-02-05T07:34:39.871575+0000","last_clean":"2024-02-05T07:34:39.871575+0000","last_became_active":"2024-02-05T07:34:04.040487+0000","last_became_peered":"2024-02-05T07:34:04.040487+0000","last_unstale":"2024-02-05T07:34:39.871575+0000","last_undegraded":"2024-02-05T07:34:39.871575+0000","last_fullsized":"2024-02-05T07:34:39.871575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:36:37.626200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735426+0000","last_change":"2024-02-05T07:33:58.005760+0000","last_active":"2024-02-05T07:34:39.735426+0000","last_peered":"2024-02-05T07:34:39.735426+0000","last_clean":"2024-02-05T07:34:39.735426+0000","last_became_active":"2024-02-05T07:33:58.005415+0000","last_became_peered":"2024-02-05T07:33:58.005415+0000","last_unstale":"2024-02-05T07:34:39.735426+0000","last_undegraded":"2024-02-05T07:34:39.735426+0000","last_fullsized":"2024-02-05T07:34:39.735426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:18:46.531096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891668+0000","last_change":"2024-02-05T07:34:00.016626+0000","last_active":"2024-02-05T07:34:39.891668+0000","last_peered":"2024-02-05T07:34:39.891668+0000","last_clean":"2024-02-05T07:34:39.891668+0000","last_became_active":"2024-02-05T07:34:00.016271+0000","last_became_peered":"2024-02-05T07:34:00.016271+0000","last_unstale":"2024-02-05T07:34:39.891668+0000","last_undegraded":"2024-02-05T07:34:39.891668+0000","last_fullsized":"2024-02-05T07:34:39.891668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:13:18.688651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.616782+0000","last_change":"2024-02-05T07:34:02.023556+0000","last_active":"2024-02-05T07:34:39.616782+0000","last_peered":"2024-02-05T07:34:39.616782+0000","last_clean":"2024-02-05T07:34:39.616782+0000","last_became_active":"2024-02-05T07:34:02.023223+0000","last_became_peered":"2024-02-05T07:34:02.023223+0000","last_unstale":"2024-02-05T07:34:39.616782+0000","last_undegraded":"2024-02-05T07:34:39.616782+0000","last_fullsized":"2024-02-05T07:34:39.616782+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:35:48.616191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734591+0000","last_change":"2024-02-05T07:34:04.038558+0000","last_active":"2024-02-05T07:34:39.734591+0000","last_peered":"2024-02-05T07:34:39.734591+0000","last_clean":"2024-02-05T07:34:39.734591+0000","last_became_active":"2024-02-05T07:34:04.038246+0000","last_became_peered":"2024-02-05T07:34:04.038246+0000","last_unstale":"2024-02-05T07:34:39.734591+0000","last_undegraded":"2024-02-05T07:34:39.734591+0000","last_fullsized":"2024-02-05T07:34:39.734591+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:43:36.952113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617324+0000","last_change":"2024-02-05T07:34:02.024571+0000","last_active":"2024-02-05T07:34:39.617324+0000","last_peered":"2024-02-05T07:34:39.617324+0000","last_clean":"2024-02-05T07:34:39.617324+0000","last_became_active":"2024-02-05T07:34:02.024254+0000","last_became_peered":"2024-02-05T07:34:02.024254+0000","last_unstale":"2024-02-05T07:34:39.617324+0000","last_undegraded":"2024-02-05T07:34:39.617324+0000","last_fullsized":"2024-02-05T07:34:39.617324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:43:35.066546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735492+0000","last_change":"2024-02-05T07:33:58.000325+0000","last_active":"2024-02-05T07:34:39.735492+0000","last_peered":"2024-02-05T07:34:39.735492+0000","last_clean":"2024-02-05T07:34:39.735492+0000","last_became_active":"2024-02-05T07:33:57.997371+0000","last_became_peered":"2024-02-05T07:33:57.997371+0000","last_unstale":"2024-02-05T07:34:39.735492+0000","last_undegraded":"2024-02-05T07:34:39.735492+0000","last_fullsized":"2024-02-05T07:34:39.735492+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:07:27.092717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":60,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619625+0000","last_change":"2024-02-05T07:34:00.267846+0000","last_active":"2024-02-05T07:34:39.619625+0000","last_peered":"2024-02-05T07:34:39.619625+0000","last_clean":"2024-02-05T07:34:39.619625+0000","last_became_active":"2024-02-05T07:34:00.267455+0000","last_became_peered":"2024-02-05T07:34:00.267455+0000","last_unstale":"2024-02-05T07:34:39.619625+0000","last_undegraded":"2024-02-05T07:34:39.619625+0000","last_fullsized":"2024-02-05T07:34:39.619625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:57:53.314787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462722+0000","last_change":"2024-02-05T07:34:04.029044+0000","last_active":"2024-02-05T07:34:40.462722+0000","last_peered":"2024-02-05T07:34:40.462722+0000","last_clean":"2024-02-05T07:34:40.462722+0000","last_became_active":"2024-02-05T07:34:04.028683+0000","last_became_peered":"2024-02-05T07:34:04.028683+0000","last_unstale":"2024-02-05T07:34:40.462722+0000","last_undegraded":"2024-02-05T07:34:40.462722+0000","last_fullsized":"2024-02-05T07:34:40.462722+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:42:00.804637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617103+0000","last_change":"2024-02-05T07:34:02.021355+0000","last_active":"2024-02-05T07:34:39.617103+0000","last_peered":"2024-02-05T07:34:39.617103+0000","last_clean":"2024-02-05T07:34:39.617103+0000","last_became_active":"2024-02-05T07:34:02.020996+0000","last_became_peered":"2024-02-05T07:34:02.020996+0000","last_unstale":"2024-02-05T07:34:39.617103+0000","last_undegraded":"2024-02-05T07:34:39.617103+0000","last_fullsized":"2024-02-05T07:34:39.617103+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:43:52.701127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724009+0000","last_change":"2024-02-05T07:33:57.996940+0000","last_active":"2024-02-05T07:34:39.724009+0000","last_peered":"2024-02-05T07:34:39.724009+0000","last_clean":"2024-02-05T07:34:39.724009+0000","last_became_active":"2024-02-05T07:33:57.996646+0000","last_became_peered":"2024-02-05T07:33:57.996646+0000","last_unstale":"2024-02-05T07:34:39.724009+0000","last_undegraded":"2024-02-05T07:34:39.724009+0000","last_fullsized":"2024-02-05T07:34:39.724009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:30:17.094356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462709+0000","last_change":"2024-02-05T07:34:00.020294+0000","last_active":"2024-02-05T07:34:40.462709+0000","last_peered":"2024-02-05T07:34:40.462709+0000","last_clean":"2024-02-05T07:34:40.462709+0000","last_became_active":"2024-02-05T07:34:00.019919+0000","last_became_peered":"2024-02-05T07:34:00.019919+0000","last_unstale":"2024-02-05T07:34:40.462709+0000","last_undegraded":"2024-02-05T07:34:40.462709+0000","last_fullsized":"2024-02-05T07:34:40.462709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:58:40.646409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871022+0000","last_change":"2024-02-05T07:34:04.037752+0000","last_active":"2024-02-05T07:34:39.871022+0000","last_peered":"2024-02-05T07:34:39.871022+0000","last_clean":"2024-02-05T07:34:39.871022+0000","last_became_active":"2024-02-05T07:34:04.037427+0000","last_became_peered":"2024-02-05T07:34:04.037427+0000","last_unstale":"2024-02-05T07:34:39.871022+0000","last_undegraded":"2024-02-05T07:34:39.871022+0000","last_fullsized":"2024-02-05T07:34:39.871022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:48:18.326146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723707+0000","last_change":"2024-02-05T07:34:01.010516+0000","last_active":"2024-02-05T07:34:39.723707+0000","last_peered":"2024-02-05T07:34:39.723707+0000","last_clean":"2024-02-05T07:34:39.723707+0000","last_became_active":"2024-02-05T07:33:56.988114+0000","last_became_peered":"2024-02-05T07:33:56.988114+0000","last_unstale":"2024-02-05T07:34:39.723707+0000","last_undegraded":"2024-02-05T07:34:39.723707+0000","last_fullsized":"2024-02-05T07:34:39.723707+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_clean_scrub_stamp":"2024-02-05T07:33:55.974370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:10:09.273216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060023400000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891588+0000","last_change":"2024-02-05T07:33:57.998412+0000","last_active":"2024-02-05T07:34:39.891588+0000","last_peered":"2024-02-05T07:34:39.891588+0000","last_clean":"2024-02-05T07:34:39.891588+0000","last_became_active":"2024-02-05T07:33:57.998116+0000","last_became_peered":"2024-02-05T07:33:57.998116+0000","last_unstale":"2024-02-05T07:34:39.891588+0000","last_undegraded":"2024-02-05T07:34:39.891588+0000","last_fullsized":"2024-02-05T07:34:39.891588+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:06:00.650203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":93,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059326+0000","last_change":"2024-02-05T07:34:00.021183+0000","last_active":"2024-02-05T07:35:08.059326+0000","last_peered":"2024-02-05T07:35:08.059326+0000","last_clean":"2024-02-05T07:35:08.059326+0000","last_became_active":"2024-02-05T07:34:00.020716+0000","last_became_peered":"2024-02-05T07:34:00.020716+0000","last_unstale":"2024-02-05T07:35:08.059326+0000","last_undegraded":"2024-02-05T07:35:08.059326+0000","last_fullsized":"2024-02-05T07:35:08.059326+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:48:23.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463210+0000","last_change":"2024-02-05T07:34:02.030209+0000","last_active":"2024-02-05T07:34:40.463210+0000","last_peered":"2024-02-05T07:34:40.463210+0000","last_clean":"2024-02-05T07:34:40.463210+0000","last_became_active":"2024-02-05T07:34:02.029888+0000","last_became_peered":"2024-02-05T07:34:02.029888+0000","last_unstale":"2024-02-05T07:34:40.463210+0000","last_undegraded":"2024-02-05T07:34:40.463210+0000","last_fullsized":"2024-02-05T07:34:40.463210+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:33:54.502182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735571+0000","last_change":"2024-02-05T07:34:04.369802+0000","last_active":"2024-02-05T07:34:39.735571+0000","last_peered":"2024-02-05T07:34:39.735571+0000","last_clean":"2024-02-05T07:34:39.735571+0000","last_became_active":"2024-02-05T07:34:04.369516+0000","last_became_peered":"2024-02-05T07:34:04.369516+0000","last_unstale":"2024-02-05T07:34:39.735571+0000","last_undegraded":"2024-02-05T07:34:39.735571+0000","last_fullsized":"2024-02-05T07:34:39.735571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:22:36.531972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619359+0000","last_change":"2024-02-05T07:33:58.001795+0000","last_active":"2024-02-05T07:34:39.619359+0000","last_peered":"2024-02-05T07:34:39.619359+0000","last_clean":"2024-02-05T07:34:39.619359+0000","last_became_active":"2024-02-05T07:33:58.001557+0000","last_became_peered":"2024-02-05T07:33:58.001557+0000","last_unstale":"2024-02-05T07:34:39.619359+0000","last_undegraded":"2024-02-05T07:34:39.619359+0000","last_fullsized":"2024-02-05T07:34:39.619359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:07:54.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.616935+0000","last_change":"2024-02-05T07:34:01.007720+0000","last_active":"2024-02-05T07:34:39.616935+0000","last_peered":"2024-02-05T07:34:39.616935+0000","last_clean":"2024-02-05T07:34:39.616935+0000","last_became_active":"2024-02-05T07:33:56.991598+0000","last_became_peered":"2024-02-05T07:33:56.991598+0000","last_unstale":"2024-02-05T07:34:39.616935+0000","last_undegraded":"2024-02-05T07:34:39.616935+0000","last_fullsized":"2024-02-05T07:34:39.616935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_clean_scrub_stamp":"2024-02-05T07:33:55.974370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:53:20.476350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088865999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.616995+0000","last_change":"2024-02-05T07:34:16.964536+0000","last_active":"2024-02-05T07:34:39.616995+0000","last_peered":"2024-02-05T07:34:39.616995+0000","last_clean":"2024-02-05T07:34:39.616995+0000","last_became_active":"2024-02-05T07:34:16.954625+0000","last_became_peered":"2024-02-05T07:34:16.954625+0000","last_unstale":"2024-02-05T07:34:39.616995+0000","last_undegraded":"2024-02-05T07:34:39.616995+0000","last_fullsized":"2024-02-05T07:34:39.616995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:35:54.143468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617008+0000","last_change":"2024-02-05T07:34:02.026320+0000","last_active":"2024-02-05T07:34:39.617008+0000","last_peered":"2024-02-05T07:34:39.617008+0000","last_clean":"2024-02-05T07:34:39.617008+0000","last_became_active":"2024-02-05T07:34:02.025101+0000","last_became_peered":"2024-02-05T07:34:02.025101+0000","last_unstale":"2024-02-05T07:34:39.617008+0000","last_undegraded":"2024-02-05T07:34:39.617008+0000","last_fullsized":"2024-02-05T07:34:39.617008+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:26:09.717222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724258+0000","last_change":"2024-02-05T07:34:04.034429+0000","last_active":"2024-02-05T07:34:39.724258+0000","last_peered":"2024-02-05T07:34:39.724258+0000","last_clean":"2024-02-05T07:34:39.724258+0000","last_became_active":"2024-02-05T07:34:04.034113+0000","last_became_peered":"2024-02-05T07:34:04.034113+0000","last_unstale":"2024-02-05T07:34:39.724258+0000","last_undegraded":"2024-02-05T07:34:39.724258+0000","last_fullsized":"2024-02-05T07:34:39.724258+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:55:46.359906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":278,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:41.718484+0000","last_change":"2024-02-05T07:33:46.958704+0000","last_active":"2024-02-05T07:34:41.718484+0000","last_peered":"2024-02-05T07:34:41.718484+0000","last_clean":"2024-02-05T07:34:41.718484+0000","last_became_active":"2024-02-05T07:33:46.958264+0000","last_became_peered":"2024-02-05T07:33:46.958264+0000","last_unstale":"2024-02-05T07:34:41.718484+0000","last_undegraded":"2024-02-05T07:34:41.718484+0000","last_fullsized":"2024-02-05T07:34:41.718484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:31:37.941215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:31:37.941215+0000","last_clean_scrub_stamp":"2024-02-05T07:31:37.941215+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:37:04.350788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871119+0000","last_change":"2024-02-05T07:33:57.997187+0000","last_active":"2024-02-05T07:34:39.871119+0000","last_peered":"2024-02-05T07:34:39.871119+0000","last_clean":"2024-02-05T07:34:39.871119+0000","last_became_active":"2024-02-05T07:33:57.996208+0000","last_became_peered":"2024-02-05T07:33:57.996208+0000","last_unstale":"2024-02-05T07:34:39.871119+0000","last_undegraded":"2024-02-05T07:34:39.871119+0000","last_fullsized":"2024-02-05T07:34:39.871119+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:23:39.654459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":67,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059713+0000","last_change":"2024-02-05T07:34:00.269699+0000","last_active":"2024-02-05T07:35:08.059713+0000","last_peered":"2024-02-05T07:35:08.059713+0000","last_clean":"2024-02-05T07:35:08.059713+0000","last_became_active":"2024-02-05T07:34:00.269325+0000","last_became_peered":"2024-02-05T07:34:00.269325+0000","last_unstale":"2024-02-05T07:35:08.059713+0000","last_undegraded":"2024-02-05T07:35:08.059713+0000","last_fullsized":"2024-02-05T07:35:08.059713+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:50:56.857301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735272+0000","last_change":"2024-02-05T07:34:02.030586+0000","last_active":"2024-02-05T07:34:39.735272+0000","last_peered":"2024-02-05T07:34:39.735272+0000","last_clean":"2024-02-05T07:34:39.735272+0000","last_became_active":"2024-02-05T07:34:02.030193+0000","last_became_peered":"2024-02-05T07:34:02.030193+0000","last_unstale":"2024-02-05T07:34:39.735272+0000","last_undegraded":"2024-02-05T07:34:39.735272+0000","last_fullsized":"2024-02-05T07:34:39.735272+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:44:09.447368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619417+0000","last_change":"2024-02-05T07:34:04.041105+0000","last_active":"2024-02-05T07:34:39.619417+0000","last_peered":"2024-02-05T07:34:39.619417+0000","last_clean":"2024-02-05T07:34:39.619417+0000","last_became_active":"2024-02-05T07:34:04.040764+0000","last_became_peered":"2024-02-05T07:34:04.040764+0000","last_unstale":"2024-02-05T07:34:39.619417+0000","last_undegraded":"2024-02-05T07:34:39.619417+0000","last_fullsized":"2024-02-05T07:34:39.619417+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:11:37.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462889+0000","last_change":"2024-02-05T07:33:58.003439+0000","last_active":"2024-02-05T07:34:40.462889+0000","last_peered":"2024-02-05T07:34:40.462889+0000","last_clean":"2024-02-05T07:34:40.462889+0000","last_became_active":"2024-02-05T07:33:58.003130+0000","last_became_peered":"2024-02-05T07:33:58.003130+0000","last_unstale":"2024-02-05T07:34:40.462889+0000","last_undegraded":"2024-02-05T07:34:40.462889+0000","last_fullsized":"2024-02-05T07:34:40.462889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:48:02.799126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":116,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:14.581988+0000","last_change":"2024-02-05T07:34:01.011414+0000","last_active":"2024-02-05T07:35:14.581988+0000","last_peered":"2024-02-05T07:35:14.581988+0000","last_clean":"2024-02-05T07:35:14.581988+0000","last_became_active":"2024-02-05T07:33:56.993501+0000","last_became_peered":"2024-02-05T07:33:56.993501+0000","last_unstale":"2024-02-05T07:35:14.581988+0000","last_undegraded":"2024-02-05T07:35:14.581988+0000","last_fullsized":"2024-02-05T07:35:14.581988+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_clean_scrub_stamp":"2024-02-05T07:33:55.974370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:24:17.351722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047533500000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463361+0000","last_change":"2024-02-05T07:34:00.013085+0000","last_active":"2024-02-05T07:34:40.463361+0000","last_peered":"2024-02-05T07:34:40.463361+0000","last_clean":"2024-02-05T07:34:40.463361+0000","last_became_active":"2024-02-05T07:34:00.012744+0000","last_became_peered":"2024-02-05T07:34:00.012744+0000","last_unstale":"2024-02-05T07:34:40.463361+0000","last_undegraded":"2024-02-05T07:34:40.463361+0000","last_fullsized":"2024-02-05T07:34:40.463361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:05:22.345969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724424+0000","last_change":"2024-02-05T07:34:02.024966+0000","last_active":"2024-02-05T07:34:39.724424+0000","last_peered":"2024-02-05T07:34:39.724424+0000","last_clean":"2024-02-05T07:34:39.724424+0000","last_became_active":"2024-02-05T07:34:02.024451+0000","last_became_peered":"2024-02-05T07:34:02.024451+0000","last_unstale":"2024-02-05T07:34:39.724424+0000","last_undegraded":"2024-02-05T07:34:39.724424+0000","last_fullsized":"2024-02-05T07:34:39.724424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:20:40.494465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619068+0000","last_change":"2024-02-05T07:34:04.367641+0000","last_active":"2024-02-05T07:34:39.619068+0000","last_peered":"2024-02-05T07:34:39.619068+0000","last_clean":"2024-02-05T07:34:39.619068+0000","last_became_active":"2024-02-05T07:34:04.367313+0000","last_became_peered":"2024-02-05T07:34:04.367313+0000","last_unstale":"2024-02-05T07:34:39.619068+0000","last_undegraded":"2024-02-05T07:34:39.619068+0000","last_fullsized":"2024-02-05T07:34:39.619068+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:31:35.209564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619743+0000","last_change":"2024-02-05T07:33:58.002752+0000","last_active":"2024-02-05T07:34:39.619743+0000","last_peered":"2024-02-05T07:34:39.619743+0000","last_clean":"2024-02-05T07:34:39.619743+0000","last_became_active":"2024-02-05T07:33:58.002457+0000","last_became_peered":"2024-02-05T07:33:58.002457+0000","last_unstale":"2024-02-05T07:34:39.619743+0000","last_undegraded":"2024-02-05T07:34:39.619743+0000","last_fullsized":"2024-02-05T07:34:39.619743+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:05:45.116433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462543+0000","last_change":"2024-02-05T07:34:00.267169+0000","last_active":"2024-02-05T07:34:40.462543+0000","last_peered":"2024-02-05T07:34:40.462543+0000","last_clean":"2024-02-05T07:34:40.462543+0000","last_became_active":"2024-02-05T07:34:00.266840+0000","last_became_peered":"2024-02-05T07:34:00.266840+0000","last_unstale":"2024-02-05T07:34:40.462543+0000","last_undegraded":"2024-02-05T07:34:40.462543+0000","last_fullsized":"2024-02-05T07:34:40.462543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:18:57.927192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619801+0000","last_change":"2024-02-05T07:34:02.032248+0000","last_active":"2024-02-05T07:34:39.619801+0000","last_peered":"2024-02-05T07:34:39.619801+0000","last_clean":"2024-02-05T07:34:39.619801+0000","last_became_active":"2024-02-05T07:34:02.031980+0000","last_became_peered":"2024-02-05T07:34:02.031980+0000","last_unstale":"2024-02-05T07:34:39.619801+0000","last_undegraded":"2024-02-05T07:34:39.619801+0000","last_fullsized":"2024-02-05T07:34:39.619801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:23:50.868885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462641+0000","last_change":"2024-02-05T07:34:04.038535+0000","last_active":"2024-02-05T07:34:40.462641+0000","last_peered":"2024-02-05T07:34:40.462641+0000","last_clean":"2024-02-05T07:34:40.462641+0000","last_became_active":"2024-02-05T07:34:04.038217+0000","last_became_peered":"2024-02-05T07:34:04.038217+0000","last_unstale":"2024-02-05T07:34:40.462641+0000","last_undegraded":"2024-02-05T07:34:40.462641+0000","last_fullsized":"2024-02-05T07:34:40.462641+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:52:04.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724189+0000","last_change":"2024-02-05T07:33:57.998024+0000","last_active":"2024-02-05T07:34:39.724189+0000","last_peered":"2024-02-05T07:34:39.724189+0000","last_clean":"2024-02-05T07:34:39.724189+0000","last_became_active":"2024-02-05T07:33:57.997657+0000","last_became_peered":"2024-02-05T07:33:57.997657+0000","last_unstale":"2024-02-05T07:34:39.724189+0000","last_undegraded":"2024-02-05T07:34:39.724189+0000","last_fullsized":"2024-02-05T07:34:39.724189+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:14:01.177060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620398+0000","last_change":"2024-02-05T07:34:00.017811+0000","last_active":"2024-02-05T07:34:39.620398+0000","last_peered":"2024-02-05T07:34:39.620398+0000","last_clean":"2024-02-05T07:34:39.620398+0000","last_became_active":"2024-02-05T07:34:00.017422+0000","last_became_peered":"2024-02-05T07:34:00.017422+0000","last_unstale":"2024-02-05T07:34:39.620398+0000","last_undegraded":"2024-02-05T07:34:39.620398+0000","last_fullsized":"2024-02-05T07:34:39.620398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:11:17.303653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059421+0000","last_change":"2024-02-05T07:34:02.029246+0000","last_active":"2024-02-05T07:35:08.059421+0000","last_peered":"2024-02-05T07:35:08.059421+0000","last_clean":"2024-02-05T07:35:08.059421+0000","last_became_active":"2024-02-05T07:34:02.028928+0000","last_became_peered":"2024-02-05T07:34:02.028928+0000","last_unstale":"2024-02-05T07:35:08.059421+0000","last_undegraded":"2024-02-05T07:35:08.059421+0000","last_fullsized":"2024-02-05T07:35:08.059421+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:59:05.616314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735134+0000","last_change":"2024-02-05T07:34:04.370494+0000","last_active":"2024-02-05T07:34:39.735134+0000","last_peered":"2024-02-05T07:34:39.735134+0000","last_clean":"2024-02-05T07:34:39.735134+0000","last_became_active":"2024-02-05T07:34:04.369638+0000","last_became_peered":"2024-02-05T07:34:04.369638+0000","last_unstale":"2024-02-05T07:34:39.735134+0000","last_undegraded":"2024-02-05T07:34:39.735134+0000","last_fullsized":"2024-02-05T07:34:39.735134+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:09:55.354271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463492+0000","last_change":"2024-02-05T07:33:57.999949+0000","last_active":"2024-02-05T07:34:40.463492+0000","last_peered":"2024-02-05T07:34:40.463492+0000","last_clean":"2024-02-05T07:34:40.463492+0000","last_became_active":"2024-02-05T07:33:57.999258+0000","last_became_peered":"2024-02-05T07:33:57.999258+0000","last_unstale":"2024-02-05T07:34:40.463492+0000","last_undegraded":"2024-02-05T07:34:40.463492+0000","last_fullsized":"2024-02-05T07:34:40.463492+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:22:07.608338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891620+0000","last_change":"2024-02-05T07:34:00.010028+0000","last_active":"2024-02-05T07:34:39.891620+0000","last_peered":"2024-02-05T07:34:39.891620+0000","last_clean":"2024-02-05T07:34:39.891620+0000","last_became_active":"2024-02-05T07:34:00.009625+0000","last_became_peered":"2024-02-05T07:34:00.009625+0000","last_unstale":"2024-02-05T07:34:39.891620+0000","last_undegraded":"2024-02-05T07:34:39.891620+0000","last_fullsized":"2024-02-05T07:34:39.891620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:04:05.562130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059377+0000","last_change":"2024-02-05T07:34:02.026329+0000","last_active":"2024-02-05T07:35:08.059377+0000","last_peered":"2024-02-05T07:35:08.059377+0000","last_clean":"2024-02-05T07:35:08.059377+0000","last_became_active":"2024-02-05T07:34:02.026034+0000","last_became_peered":"2024-02-05T07:34:02.026034+0000","last_unstale":"2024-02-05T07:35:08.059377+0000","last_undegraded":"2024-02-05T07:35:08.059377+0000","last_fullsized":"2024-02-05T07:35:08.059377+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:46:32.527993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891465+0000","last_change":"2024-02-05T07:34:04.038303+0000","last_active":"2024-02-05T07:34:39.891465+0000","last_peered":"2024-02-05T07:34:39.891465+0000","last_clean":"2024-02-05T07:34:39.891465+0000","last_became_active":"2024-02-05T07:34:04.038019+0000","last_became_peered":"2024-02-05T07:34:04.038019+0000","last_unstale":"2024-02-05T07:34:39.891465+0000","last_undegraded":"2024-02-05T07:34:39.891465+0000","last_fullsized":"2024-02-05T07:34:39.891465+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:15:42.606783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871274+0000","last_change":"2024-02-05T07:33:57.993348+0000","last_active":"2024-02-05T07:34:39.871274+0000","last_peered":"2024-02-05T07:34:39.871274+0000","last_clean":"2024-02-05T07:34:39.871274+0000","last_became_active":"2024-02-05T07:33:57.993051+0000","last_became_peered":"2024-02-05T07:33:57.993051+0000","last_unstale":"2024-02-05T07:34:39.871274+0000","last_undegraded":"2024-02-05T07:34:39.871274+0000","last_fullsized":"2024-02-05T07:34:39.871274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:26:31.251993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870750+0000","last_change":"2024-02-05T07:34:00.266782+0000","last_active":"2024-02-05T07:34:39.870750+0000","last_peered":"2024-02-05T07:34:39.870750+0000","last_clean":"2024-02-05T07:34:39.870750+0000","last_became_active":"2024-02-05T07:34:00.266479+0000","last_became_peered":"2024-02-05T07:34:00.266479+0000","last_unstale":"2024-02-05T07:34:39.870750+0000","last_undegraded":"2024-02-05T07:34:39.870750+0000","last_fullsized":"2024-02-05T07:34:39.870750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:21:34.761450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891300+0000","last_change":"2024-02-05T07:34:02.027486+0000","last_active":"2024-02-05T07:34:39.891300+0000","last_peered":"2024-02-05T07:34:39.891300+0000","last_clean":"2024-02-05T07:34:39.891300+0000","last_became_active":"2024-02-05T07:34:02.027180+0000","last_became_peered":"2024-02-05T07:34:02.027180+0000","last_unstale":"2024-02-05T07:34:39.891300+0000","last_undegraded":"2024-02-05T07:34:39.891300+0000","last_fullsized":"2024-02-05T07:34:39.891300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:14:19.617186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724010+0000","last_change":"2024-02-05T07:34:04.033403+0000","last_active":"2024-02-05T07:34:39.724010+0000","last_peered":"2024-02-05T07:34:39.724010+0000","last_clean":"2024-02-05T07:34:39.724010+0000","last_became_active":"2024-02-05T07:34:04.033126+0000","last_became_peered":"2024-02-05T07:34:04.033126+0000","last_unstale":"2024-02-05T07:34:39.724010+0000","last_undegraded":"2024-02-05T07:34:39.724010+0000","last_fullsized":"2024-02-05T07:34:39.724010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:11:39.907031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059625+0000","last_change":"2024-02-05T07:34:02.027667+0000","last_active":"2024-02-05T07:35:08.059625+0000","last_peered":"2024-02-05T07:35:08.059625+0000","last_clean":"2024-02-05T07:35:08.059625+0000","last_became_active":"2024-02-05T07:34:02.024417+0000","last_became_peered":"2024-02-05T07:34:02.024417+0000","last_unstale":"2024-02-05T07:35:08.059625+0000","last_undegraded":"2024-02-05T07:35:08.059625+0000","last_fullsized":"2024-02-05T07:35:08.059625+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:33:35.007406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870728+0000","last_change":"2024-02-05T07:33:57.997642+0000","last_active":"2024-02-05T07:34:39.870728+0000","last_peered":"2024-02-05T07:34:39.870728+0000","last_clean":"2024-02-05T07:34:39.870728+0000","last_became_active":"2024-02-05T07:33:57.997361+0000","last_became_peered":"2024-02-05T07:33:57.997361+0000","last_unstale":"2024-02-05T07:34:39.870728+0000","last_undegraded":"2024-02-05T07:34:39.870728+0000","last_fullsized":"2024-02-05T07:34:39.870728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:32:26.163667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620140+0000","last_change":"2024-02-05T07:34:00.015169+0000","last_active":"2024-02-05T07:34:39.620140+0000","last_peered":"2024-02-05T07:34:39.620140+0000","last_clean":"2024-02-05T07:34:39.620140+0000","last_became_active":"2024-02-05T07:34:00.014865+0000","last_became_peered":"2024-02-05T07:34:00.014865+0000","last_unstale":"2024-02-05T07:34:39.620140+0000","last_undegraded":"2024-02-05T07:34:39.620140+0000","last_fullsized":"2024-02-05T07:34:39.620140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:34:39.394930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871216+0000","last_change":"2024-02-05T07:34:04.368042+0000","last_active":"2024-02-05T07:34:39.871216+0000","last_peered":"2024-02-05T07:34:39.871216+0000","last_clean":"2024-02-05T07:34:39.871216+0000","last_became_active":"2024-02-05T07:34:04.367707+0000","last_became_peered":"2024-02-05T07:34:04.367707+0000","last_unstale":"2024-02-05T07:34:39.871216+0000","last_undegraded":"2024-02-05T07:34:39.871216+0000","last_fullsized":"2024-02-05T07:34:39.871216+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:21:16.757195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724401+0000","last_change":"2024-02-05T07:34:02.024725+0000","last_active":"2024-02-05T07:34:39.724401+0000","last_peered":"2024-02-05T07:34:39.724401+0000","last_clean":"2024-02-05T07:34:39.724401+0000","last_became_active":"2024-02-05T07:34:02.023947+0000","last_became_peered":"2024-02-05T07:34:02.023947+0000","last_unstale":"2024-02-05T07:34:39.724401+0000","last_undegraded":"2024-02-05T07:34:39.724401+0000","last_fullsized":"2024-02-05T07:34:39.724401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:18:17.899108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891394+0000","last_change":"2024-02-05T07:33:58.003615+0000","last_active":"2024-02-05T07:34:39.891394+0000","last_peered":"2024-02-05T07:34:39.891394+0000","last_clean":"2024-02-05T07:34:39.891394+0000","last_became_active":"2024-02-05T07:33:58.003068+0000","last_became_peered":"2024-02-05T07:33:58.003068+0000","last_unstale":"2024-02-05T07:34:39.891394+0000","last_undegraded":"2024-02-05T07:34:39.891394+0000","last_fullsized":"2024-02-05T07:34:39.891394+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:13:46.594789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892177+0000","last_change":"2024-02-05T07:34:00.014686+0000","last_active":"2024-02-05T07:34:39.892177+0000","last_peered":"2024-02-05T07:34:39.892177+0000","last_clean":"2024-02-05T07:34:39.892177+0000","last_became_active":"2024-02-05T07:34:00.014313+0000","last_became_peered":"2024-02-05T07:34:00.014313+0000","last_unstale":"2024-02-05T07:34:39.892177+0000","last_undegraded":"2024-02-05T07:34:39.892177+0000","last_fullsized":"2024-02-05T07:34:39.892177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:34:55.487862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617226+0000","last_change":"2024-02-05T07:34:04.035996+0000","last_active":"2024-02-05T07:34:39.617226+0000","last_peered":"2024-02-05T07:34:39.617226+0000","last_clean":"2024-02-05T07:34:39.617226+0000","last_became_active":"2024-02-05T07:34:04.035256+0000","last_became_peered":"2024-02-05T07:34:04.035256+0000","last_unstale":"2024-02-05T07:34:39.617226+0000","last_undegraded":"2024-02-05T07:34:39.617226+0000","last_fullsized":"2024-02-05T07:34:39.617226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:09:19.804514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620005+0000","last_change":"2024-02-05T07:34:02.023289+0000","last_active":"2024-02-05T07:34:39.620005+0000","last_peered":"2024-02-05T07:34:39.620005+0000","last_clean":"2024-02-05T07:34:39.620005+0000","last_became_active":"2024-02-05T07:34:02.022920+0000","last_became_peered":"2024-02-05T07:34:02.022920+0000","last_unstale":"2024-02-05T07:34:39.620005+0000","last_undegraded":"2024-02-05T07:34:39.620005+0000","last_fullsized":"2024-02-05T07:34:39.620005+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:00:28.904996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619443+0000","last_change":"2024-02-05T07:33:57.997980+0000","last_active":"2024-02-05T07:34:39.619443+0000","last_peered":"2024-02-05T07:34:39.619443+0000","last_clean":"2024-02-05T07:34:39.619443+0000","last_became_active":"2024-02-05T07:33:57.997673+0000","last_became_peered":"2024-02-05T07:33:57.997673+0000","last_unstale":"2024-02-05T07:34:39.619443+0000","last_undegraded":"2024-02-05T07:34:39.619443+0000","last_fullsized":"2024-02-05T07:34:39.619443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:34:57.570197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892087+0000","last_change":"2024-02-05T07:34:00.014699+0000","last_active":"2024-02-05T07:34:39.892087+0000","last_peered":"2024-02-05T07:34:39.892087+0000","last_clean":"2024-02-05T07:34:39.892087+0000","last_became_active":"2024-02-05T07:34:00.014140+0000","last_became_peered":"2024-02-05T07:34:00.014140+0000","last_unstale":"2024-02-05T07:34:39.892087+0000","last_undegraded":"2024-02-05T07:34:39.892087+0000","last_fullsized":"2024-02-05T07:34:39.892087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:32:34.659965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892338+0000","last_change":"2024-02-05T07:34:04.036556+0000","last_active":"2024-02-05T07:34:39.892338+0000","last_peered":"2024-02-05T07:34:39.892338+0000","last_clean":"2024-02-05T07:34:39.892338+0000","last_became_active":"2024-02-05T07:34:04.036254+0000","last_became_peered":"2024-02-05T07:34:04.036254+0000","last_unstale":"2024-02-05T07:34:39.892338+0000","last_undegraded":"2024-02-05T07:34:39.892338+0000","last_fullsized":"2024-02-05T07:34:39.892338+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:15:14.107732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":49,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:08.059328+0000","last_change":"2024-02-05T07:34:02.025976+0000","last_active":"2024-02-05T07:35:08.059328+0000","last_peered":"2024-02-05T07:35:08.059328+0000","last_clean":"2024-02-05T07:35:08.059328+0000","last_became_active":"2024-02-05T07:34:02.025623+0000","last_became_peered":"2024-02-05T07:34:02.025623+0000","last_unstale":"2024-02-05T07:35:08.059328+0000","last_undegraded":"2024-02-05T07:35:08.059328+0000","last_fullsized":"2024-02-05T07:35:08.059328+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:28:28.422092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871745+0000","last_change":"2024-02-05T07:33:58.003609+0000","last_active":"2024-02-05T07:34:39.871745+0000","last_peered":"2024-02-05T07:34:39.871745+0000","last_clean":"2024-02-05T07:34:39.871745+0000","last_became_active":"2024-02-05T07:33:58.003173+0000","last_became_peered":"2024-02-05T07:33:58.003173+0000","last_unstale":"2024-02-05T07:34:39.871745+0000","last_undegraded":"2024-02-05T07:34:39.871745+0000","last_fullsized":"2024-02-05T07:34:39.871745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:41:41.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891847+0000","last_change":"2024-02-05T07:34:00.016560+0000","last_active":"2024-02-05T07:34:39.891847+0000","last_peered":"2024-02-05T07:34:39.891847+0000","last_clean":"2024-02-05T07:34:39.891847+0000","last_became_active":"2024-02-05T07:34:00.016214+0000","last_became_peered":"2024-02-05T07:34:00.016214+0000","last_unstale":"2024-02-05T07:34:39.891847+0000","last_undegraded":"2024-02-05T07:34:39.891847+0000","last_fullsized":"2024-02-05T07:34:39.891847+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:55:28.194806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168596,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":936896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.71699999999999997}]},{"osd":1,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.64200000000000002}]},{"osd":2,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.61899999999999999}]},{"osd":3,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55500000000000005}]},{"osd":4,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.435}]},{"osd":5,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.46400000000000002}]},{"osd":6,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.501},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.52800000000000002}]}]},{"osd":6,"up_from":48,"seq":206158430233,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32544,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712608,"statfs":{"total":95995035648,"available":95961710592,"internally_reserved":0,"allocated":1400832,"data_stored":934586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.44500000000000001}]},{"osd":1,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.46200000000000002}]},{"osd":2,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65400000000000003}]},{"osd":4,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69499999999999995}]},{"osd":5,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.66800000000000004}]},{"osd":7,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":345550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.371,"15min":0.33300000000000002},"min":{"1min":0.27100000000000002,"5min":0.157,"15min":0.157},"max":{"1min":0.53400000000000003,"5min":0.628,"15min":0.628},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.34399999999999997},"min":{"1min":0.26500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.505,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.37}]},{"osd":2,"last update":"Mon Feb 5 07:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.40899999999999997,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.189,"15min":0.189},"max":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.41199999999999998},"min":{"1min":0.27400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.61199999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.32400000000000001}]},{"osd":3,"last update":"Mon Feb 5 07:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47699999999999998,"15min":0.47499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44900000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.34300000000000003}]},{"osd":4,"last update":"Mon Feb 5 07:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.621}]},{"osd":5,"last update":"Mon Feb 5 07:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59099999999999997}]},{"osd":6,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.504}]},{"osd":7,"last update":"Mon Feb 5 07:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65400000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1429504,"data_stored":936639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 5 07:34:14 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.36399999999999999,"15min":0.33600000000000002},"min":{"1min":0.27300000000000002,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.67700000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.40999999999999998,"15min":0.38200000000000001},"min":{"1min":0.35699999999999998,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.60099999999999998,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.377}]},{"osd":2,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52500000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.35399999999999998}]},{"osd":3,"last update":"Mon Feb 5 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.27200000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48999999999999999}]},{"osd":4,"last update":"Mon Feb 5 07:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.77500000000000002}]},{"osd":5,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.86599999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.622},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.89500000000000002}]},{"osd":7,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.93000000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":811008,"data_stored":344649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.377,"15min":0.36599999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.54100000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.35799999999999998}]},{"osd":1,"last update":"Mon Feb 5 07:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.621,"15min":0.621},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.40899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.32000000000000001}]},{"osd":3,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.27300000000000002}]},{"osd":4,"last update":"Mon Feb 5 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68000000000000005}]},{"osd":5,"last update":"Mon Feb 5 07:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81899999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84799999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.61499999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.745},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69299999999999995}]},{"osd":7,"last update":"Mon Feb 5 07:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.52500000000000002}]}]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":345983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.45800000000000002,"15min":0.433},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":5.8949999999999996,"5min":5.8949999999999996,"15min":5.8949999999999996},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.44,"5min":0.437,"15min":0.442},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46899999999999997,"15min":0.47899999999999998},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.223},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.46200000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.1150000000000002,"5min":3.1150000000000002,"15min":3.1150000000000002},"last":0.42899999999999999}]},{"osd":2,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48099999999999998,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":5.6529999999999996,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.29999999999999999}]},{"osd":4,"last update":"Mon Feb 5 07:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.624}]},{"osd":5,"last update":"Mon Feb 5 07:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.66500000000000004,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.5979999999999999,"5min":5.5979999999999999,"15min":5.5979999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.60299999999999998}]},{"osd":6,"last update":"Mon Feb 5 07:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.81599999999999995}]},{"osd":7,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.6879999999999997,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.76600000000000001}]}]},{"osd":4,"up_from":32,"seq":137438953509,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":345208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68700000000000006}]},{"osd":1,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.627},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88900000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76200000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.57999999999999996}]},{"osd":3,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.70999999999999996}]},{"osd":5,"last update":"Mon Feb 5 07:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65100000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.433,"5min":0.436,"15min":0.436},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Mon Feb 5 07:34:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.74299999999999999}]},{"osd":7,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.80600000000000005}]}]},{"osd":5,"up_from":40,"seq":171798691871,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":815104,"data_stored":344653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66900000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.61199999999999999}]},{"osd":1,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.72099999999999997,"5min":0.754,"15min":0.754},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.67800000000000005}]},{"osd":2,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56699999999999995,"15min":0.57599999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.64800000000000002}]},{"osd":3,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.78600000000000003}]},{"osd":4,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.48399999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50900000000000001}]},{"osd":7,"last update":"Mon Feb 5 07:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.56999999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-05T07:35:21.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-05T07:35:21.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:20 smithi050 ceph-mon[33492]: from='client.14727 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:35:21.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:20 smithi050 ceph-mon[33492]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:21.813 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 podman[79022]: 2024-02-05 07:35:21.595223709 +0000 UTC m=+0.226250302 container create e67eb5621b2dec09a069419af56f4908dd82e288241d4a7641b840cd9d198f9b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:22.157 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:35:22.215 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:22.215 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:22.215 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 podman[79022]: 2024-02-05 07:35:21.923199277 +0000 UTC m=+0.554225867 container init e67eb5621b2dec09a069419af56f4908dd82e288241d4a7641b840cd9d198f9b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:22.215 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 podman[79022]: 2024-02-05 07:35:21.928464671 +0000 UTC m=+0.559491261 container start e67eb5621b2dec09a069419af56f4908dd82e288241d4a7641b840cd9d198f9b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:22.215 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.940Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-05T07:35:22.215 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.940Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.941Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.942Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.967Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.967Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.970Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:21.970Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:22 smithi043 bash[79022]: e67eb5621b2dec09a069419af56f4908dd82e288241d4a7641b840cd9d198f9b 2024-02-05T07:35:22.216 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:22 smithi043 systemd[1]: Started Ceph alertmanager.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:35:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:22.543 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:35:22] ENGINE Bus STOPPING 2024-02-05T07:35:22.544 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:35:22] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:35:22.544 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:35:22] ENGINE Bus STOPPED 2024-02-05T07:35:22.544 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:35:22] ENGINE Bus STARTING 2024-02-05T07:35:22.544 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:35:22] ENGINE Serving on http://:::9283 2024-02-05T07:35:22.544 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:22 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:35:22] ENGINE Bus STARTED 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:35:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:35:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:35:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:23.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.846+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/137884545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 msgr2=0x7fdd681058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:23.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.846+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/137884545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd681058f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdd580098f0 tx=0x7fdd58009c00 comp rx=0 tx=0).stop 2024-02-05T07:35:23.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.846+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/137884545 shutdown_connections 2024-02-05T07:35:23.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.846+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/137884545 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd68105fd0 0x7fdd6810dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.846+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/137884545 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd68100ef0 0x7fdd681012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.846+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/137884545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd681058f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.846+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/137884545 >> 172.21.15.43:0/137884545 conn(0x7fdd68077d50 msgr2=0x7fdd68078160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/137884545 shutdown_connections 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/137884545 wait complete. 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 Processor -- start 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 -- start start 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd68100ef0 0x7fdd6819d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd6819db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd68105fd0 0x7fdd68197700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68197eb0 con 0x7fdd681018a0 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd68198020 con 0x7fdd68100ef0 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.847+0000 7fdd6e3e7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd681981e0 con 0x7fdd68105fd0 2024-02-05T07:35:23.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd6819db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd6819db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51910/0 (socket says 172.21.15.43:51910) 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 -- 172.21.15.43:0/1264259473 learned_addr learned my addr 172.21.15.43:0/1264259473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd6c984700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd68105fd0 0x7fdd68197700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd67fff700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd68100ef0 0x7fdd6819d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 -- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd68105fd0 msgr2=0x7fdd68197700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd68105fd0 0x7fdd68197700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 -- 172.21.15.43:0/1264259473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd68100ef0 msgr2=0x7fdd6819d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd68100ef0 0x7fdd6819d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 -- 172.21.15.43:0/1264259473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60009d00 con 0x7fdd681018a0 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd67fff700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd68100ef0 0x7fdd6819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd677fe700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd6819db20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009230 tx=0x7fdd5800f990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd657fa700 1 -- 172.21.15.43:0/1264259473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd5801c070 con 0x7fdd681018a0 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd657fa700 1 -- 172.21.15.43:0/1264259473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdd5800b9e0 con 0x7fdd681018a0 2024-02-05T07:35:23.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd58009520 con 0x7fdd681018a0 2024-02-05T07:35:23.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd657fa700 1 -- 172.21.15.43:0/1264259473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58003680 con 0x7fdd681018a0 2024-02-05T07:35:23.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.848+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd681a4390 con 0x7fdd681018a0 2024-02-05T07:35:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.849+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6810a300 con 0x7fdd681018a0 2024-02-05T07:35:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.850+0000 7fdd657fa700 1 -- 172.21.15.43:0/1264259473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fdd58020030 con 0x7fdd681018a0 2024-02-05T07:35:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.850+0000 7fdd657fa700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdd5006dac0 0x7fdd5006ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.850+0000 7fdd657fa700 1 -- 172.21.15.43:0/1264259473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7fdd5808e640 con 0x7fdd681018a0 2024-02-05T07:35:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.850+0000 7fdd67fff700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdd5006dac0 0x7fdd5006ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:23.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.851+0000 7fdd67fff700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdd5006dac0 0x7fdd5006ff80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c011490 tx=0x7fdd5c009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:23.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:23.855+0000 7fdd657fa700 1 -- 172.21.15.43:0/1264259473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd58057ad0 con 0x7fdd681018a0 2024-02-05T07:35:24.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.056+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 --> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdd680689e0 con 0x7fdd5006dac0 2024-02-05T07:35:24.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.067+0000 7fdd657fa700 1 -- 172.21.15.43:0/1264259473 <== mgr.24485 v2:172.21.15.43:6800/847956893 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+384062 (secure 0 0 0) 0x7fdd680689e0 con 0x7fdd5006dac0 2024-02-05T07:35:24.073 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdd5006dac0 msgr2=0x7fdd5006ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdd5006dac0 0x7fdd5006ff80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c011490 tx=0x7fdd5c009260 comp rx=0 tx=0).stop 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 msgr2=0x7fdd6819db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd6819db20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009230 tx=0x7fdd5800f990 comp rx=0 tx=0).stop 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 shutdown_connections 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdd5006dac0 0x7fdd5006ff80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdd68105fd0 0x7fdd68197700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdd68100ef0 0x7fdd6819d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 --2- 172.21.15.43:0/1264259473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd681018a0 0x7fdd6819db20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 >> 172.21.15.43:0/1264259473 conn(0x7fdd68077d50 msgr2=0x7fdd6806a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 shutdown_connections 2024-02-05T07:35:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:24.070+0000 7fdd6e3e7700 1 -- 172.21.15.43:0/1264259473 wait complete. 2024-02-05T07:35:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-02-05T07:35:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:24 smithi043 ceph-mon[29879]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T07:35:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:24 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:35:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:24 smithi043 ceph-mon[29879]: Deploying daemon grafana.a on smithi050 2024-02-05T07:35:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:24 smithi043 ceph-mon[41444]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T07:35:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:24 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:35:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:24 smithi043 ceph-mon[41444]: Deploying daemon grafana.a on smithi050 2024-02-05T07:35:24.314 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:23.943Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000396414s 2024-02-05T07:35:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:24 smithi050 ceph-mon[33492]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-05T07:35:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:24 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-05T07:35:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:24 smithi050 ceph-mon[33492]: Deploying daemon grafana.a on smithi050 2024-02-05T07:35:24.925 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-02-05T07:35:23.638231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596316,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1009,"num_read_kb":855,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1771544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":245168,"kb_used_data":8220,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716048,"statfs":{"total":767960285184,"available":767709233152,"internally_reserved":0,"allocated":8417280,"data_stored":4534164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003440"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891390+0000","last_change":"2024-02-05T07:34:04.031923+0000","last_active":"2024-02-05T07:34:39.891390+0000","last_peered":"2024-02-05T07:34:39.891390+0000","last_clean":"2024-02-05T07:34:39.891390+0000","last_became_active":"2024-02-05T07:34:04.031632+0000","last_became_peered":"2024-02-05T07:34:04.031632+0000","last_unstale":"2024-02-05T07:34:39.891390+0000","last_undegraded":"2024-02-05T07:34:39.891390+0000","last_fullsized":"2024-02-05T07:34:39.891390+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:25:08.459570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619167+0000","last_change":"2024-02-05T07:34:02.027064+0000","last_active":"2024-02-05T07:34:39.619167+0000","last_peered":"2024-02-05T07:34:39.619167+0000","last_clean":"2024-02-05T07:34:39.619167+0000","last_became_active":"2024-02-05T07:34:02.026763+0000","last_became_peered":"2024-02-05T07:34:02.026763+0000","last_unstale":"2024-02-05T07:34:39.619167+0000","last_undegraded":"2024-02-05T07:34:39.619167+0000","last_fullsized":"2024-02-05T07:34:39.619167+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:35:09.525261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463550+0000","last_change":"2024-02-05T07:33:57.996546+0000","last_active":"2024-02-05T07:34:40.463550+0000","last_peered":"2024-02-05T07:34:40.463550+0000","last_clean":"2024-02-05T07:34:40.463550+0000","last_became_active":"2024-02-05T07:33:57.996078+0000","last_became_peered":"2024-02-05T07:33:57.996078+0000","last_unstale":"2024-02-05T07:34:40.463550+0000","last_undegraded":"2024-02-05T07:34:40.463550+0000","last_fullsized":"2024-02-05T07:34:40.463550+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:33:38.836788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891310+0000","last_change":"2024-02-05T07:34:00.014624+0000","last_active":"2024-02-05T07:34:39.891310+0000","last_peered":"2024-02-05T07:34:39.891310+0000","last_clean":"2024-02-05T07:34:39.891310+0000","last_became_active":"2024-02-05T07:34:00.014244+0000","last_became_peered":"2024-02-05T07:34:00.014244+0000","last_unstale":"2024-02-05T07:34:39.891310+0000","last_undegraded":"2024-02-05T07:34:39.891310+0000","last_fullsized":"2024-02-05T07:34:39.891310+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:15:10.317337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.872076+0000","last_change":"2024-02-05T07:34:04.369467+0000","last_active":"2024-02-05T07:34:39.872076+0000","last_peered":"2024-02-05T07:34:39.872076+0000","last_clean":"2024-02-05T07:34:39.872076+0000","last_became_active":"2024-02-05T07:34:04.369121+0000","last_became_peered":"2024-02-05T07:34:04.369121+0000","last_unstale":"2024-02-05T07:34:39.872076+0000","last_undegraded":"2024-02-05T07:34:39.872076+0000","last_fullsized":"2024-02-05T07:34:39.872076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:36:08.873593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891313+0000","last_change":"2024-02-05T07:34:02.024818+0000","last_active":"2024-02-05T07:34:39.891313+0000","last_peered":"2024-02-05T07:34:39.891313+0000","last_clean":"2024-02-05T07:34:39.891313+0000","last_became_active":"2024-02-05T07:34:02.024497+0000","last_became_peered":"2024-02-05T07:34:02.024497+0000","last_unstale":"2024-02-05T07:34:39.891313+0000","last_undegraded":"2024-02-05T07:34:39.891313+0000","last_fullsized":"2024-02-05T07:34:39.891313+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:25:48.725488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871897+0000","last_change":"2024-02-05T07:33:57.996910+0000","last_active":"2024-02-05T07:34:39.871897+0000","last_peered":"2024-02-05T07:34:39.871897+0000","last_clean":"2024-02-05T07:34:39.871897+0000","last_became_active":"2024-02-05T07:33:57.996609+0000","last_became_peered":"2024-02-05T07:33:57.996609+0000","last_unstale":"2024-02-05T07:34:39.871897+0000","last_undegraded":"2024-02-05T07:34:39.871897+0000","last_fullsized":"2024-02-05T07:34:39.871897+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:57:51.437191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871369+0000","last_change":"2024-02-05T07:34:00.018001+0000","last_active":"2024-02-05T07:34:39.871369+0000","last_peered":"2024-02-05T07:34:39.871369+0000","last_clean":"2024-02-05T07:34:39.871369+0000","last_became_active":"2024-02-05T07:34:00.017656+0000","last_became_peered":"2024-02-05T07:34:00.017656+0000","last_unstale":"2024-02-05T07:34:39.871369+0000","last_undegraded":"2024-02-05T07:34:39.871369+0000","last_fullsized":"2024-02-05T07:34:39.871369+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:01:53.164886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462878+0000","last_change":"2024-02-05T07:34:04.042609+0000","last_active":"2024-02-05T07:34:40.462878+0000","last_peered":"2024-02-05T07:34:40.462878+0000","last_clean":"2024-02-05T07:34:40.462878+0000","last_became_active":"2024-02-05T07:34:04.042264+0000","last_became_peered":"2024-02-05T07:34:04.042264+0000","last_unstale":"2024-02-05T07:34:40.462878+0000","last_undegraded":"2024-02-05T07:34:40.462878+0000","last_fullsized":"2024-02-05T07:34:40.462878+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:51:41.134652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723801+0000","last_change":"2024-02-05T07:34:02.025982+0000","last_active":"2024-02-05T07:34:39.723801+0000","last_peered":"2024-02-05T07:34:39.723801+0000","last_clean":"2024-02-05T07:34:39.723801+0000","last_became_active":"2024-02-05T07:34:02.025683+0000","last_became_peered":"2024-02-05T07:34:02.025683+0000","last_unstale":"2024-02-05T07:34:39.723801+0000","last_undegraded":"2024-02-05T07:34:39.723801+0000","last_fullsized":"2024-02-05T07:34:39.723801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:31:36.871597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724093+0000","last_change":"2024-02-05T07:33:57.998743+0000","last_active":"2024-02-05T07:34:39.724093+0000","last_peered":"2024-02-05T07:34:39.724093+0000","last_clean":"2024-02-05T07:34:39.724093+0000","last_became_active":"2024-02-05T07:33:57.998414+0000","last_became_peered":"2024-02-05T07:33:57.998414+0000","last_unstale":"2024-02-05T07:34:39.724093+0000","last_undegraded":"2024-02-05T07:34:39.724093+0000","last_fullsized":"2024-02-05T07:34:39.724093+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:29:42.546996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891585+0000","last_change":"2024-02-05T07:34:00.012151+0000","last_active":"2024-02-05T07:34:39.891585+0000","last_peered":"2024-02-05T07:34:39.891585+0000","last_clean":"2024-02-05T07:34:39.891585+0000","last_became_active":"2024-02-05T07:34:00.011747+0000","last_became_peered":"2024-02-05T07:34:00.011747+0000","last_unstale":"2024-02-05T07:34:39.891585+0000","last_undegraded":"2024-02-05T07:34:39.891585+0000","last_fullsized":"2024-02-05T07:34:39.891585+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:25:46.642968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723514+0000","last_change":"2024-02-05T07:34:04.039520+0000","last_active":"2024-02-05T07:34:39.723514+0000","last_peered":"2024-02-05T07:34:39.723514+0000","last_clean":"2024-02-05T07:34:39.723514+0000","last_became_active":"2024-02-05T07:34:04.039182+0000","last_became_peered":"2024-02-05T07:34:04.039182+0000","last_unstale":"2024-02-05T07:34:39.723514+0000","last_undegraded":"2024-02-05T07:34:39.723514+0000","last_fullsized":"2024-02-05T07:34:39.723514+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:04:39.438940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734773+0000","last_change":"2024-02-05T07:34:02.029675+0000","last_active":"2024-02-05T07:34:39.734773+0000","last_peered":"2024-02-05T07:34:39.734773+0000","last_clean":"2024-02-05T07:34:39.734773+0000","last_became_active":"2024-02-05T07:34:02.029447+0000","last_became_peered":"2024-02-05T07:34:02.029447+0000","last_unstale":"2024-02-05T07:34:39.734773+0000","last_undegraded":"2024-02-05T07:34:39.734773+0000","last_fullsized":"2024-02-05T07:34:39.734773+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:37:24.173720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723938+0000","last_change":"2024-02-05T07:33:58.000439+0000","last_active":"2024-02-05T07:34:39.723938+0000","last_peered":"2024-02-05T07:34:39.723938+0000","last_clean":"2024-02-05T07:34:39.723938+0000","last_became_active":"2024-02-05T07:33:57.999986+0000","last_became_peered":"2024-02-05T07:33:57.999986+0000","last_unstale":"2024-02-05T07:34:39.723938+0000","last_undegraded":"2024-02-05T07:34:39.723938+0000","last_fullsized":"2024-02-05T07:34:39.723938+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:59:51.789954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'5","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891825+0000","last_change":"2024-02-05T07:34:00.015319+0000","last_active":"2024-02-05T07:34:39.891825+0000","last_peered":"2024-02-05T07:34:39.891825+0000","last_clean":"2024-02-05T07:34:39.891825+0000","last_became_active":"2024-02-05T07:34:00.014964+0000","last_became_peered":"2024-02-05T07:34:00.014964+0000","last_unstale":"2024-02-05T07:34:39.891825+0000","last_undegraded":"2024-02-05T07:34:39.891825+0000","last_fullsized":"2024-02-05T07:34:39.891825+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:54:35.412027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892217+0000","last_change":"2024-02-05T07:34:04.368376+0000","last_active":"2024-02-05T07:34:39.892217+0000","last_peered":"2024-02-05T07:34:39.892217+0000","last_clean":"2024-02-05T07:34:39.892217+0000","last_became_active":"2024-02-05T07:34:04.367455+0000","last_became_peered":"2024-02-05T07:34:04.367455+0000","last_unstale":"2024-02-05T07:34:39.892217+0000","last_undegraded":"2024-02-05T07:34:39.892217+0000","last_fullsized":"2024-02-05T07:34:39.892217+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:41:02.971468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619675+0000","last_change":"2024-02-05T07:34:02.024447+0000","last_active":"2024-02-05T07:34:39.619675+0000","last_peered":"2024-02-05T07:34:39.619675+0000","last_clean":"2024-02-05T07:34:39.619675+0000","last_became_active":"2024-02-05T07:34:02.024123+0000","last_became_peered":"2024-02-05T07:34:02.024123+0000","last_unstale":"2024-02-05T07:34:39.619675+0000","last_undegraded":"2024-02-05T07:34:39.619675+0000","last_fullsized":"2024-02-05T07:34:39.619675+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:57:57.972408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463231+0000","last_change":"2024-02-05T07:33:58.001920+0000","last_active":"2024-02-05T07:34:40.463231+0000","last_peered":"2024-02-05T07:34:40.463231+0000","last_clean":"2024-02-05T07:34:40.463231+0000","last_became_active":"2024-02-05T07:33:58.001592+0000","last_became_peered":"2024-02-05T07:33:58.001592+0000","last_unstale":"2024-02-05T07:34:40.463231+0000","last_undegraded":"2024-02-05T07:34:40.463231+0000","last_fullsized":"2024-02-05T07:34:40.463231+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:33:33.448769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617693+0000","last_change":"2024-02-05T07:34:00.016711+0000","last_active":"2024-02-05T07:34:39.617693+0000","last_peered":"2024-02-05T07:34:39.617693+0000","last_clean":"2024-02-05T07:34:39.617693+0000","last_became_active":"2024-02-05T07:34:00.016363+0000","last_became_peered":"2024-02-05T07:34:00.016363+0000","last_unstale":"2024-02-05T07:34:39.617693+0000","last_undegraded":"2024-02-05T07:34:39.617693+0000","last_fullsized":"2024-02-05T07:34:39.617693+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:44:11.434929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871568+0000","last_change":"2024-02-05T07:34:04.368183+0000","last_active":"2024-02-05T07:34:39.871568+0000","last_peered":"2024-02-05T07:34:39.871568+0000","last_clean":"2024-02-05T07:34:39.871568+0000","last_became_active":"2024-02-05T07:34:04.367787+0000","last_became_peered":"2024-02-05T07:34:04.367787+0000","last_unstale":"2024-02-05T07:34:39.871568+0000","last_undegraded":"2024-02-05T07:34:39.871568+0000","last_fullsized":"2024-02-05T07:34:39.871568+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:13:56.195144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891661+0000","last_change":"2024-02-05T07:34:02.027658+0000","last_active":"2024-02-05T07:34:39.891661+0000","last_peered":"2024-02-05T07:34:39.891661+0000","last_clean":"2024-02-05T07:34:39.891661+0000","last_became_active":"2024-02-05T07:34:02.027365+0000","last_became_peered":"2024-02-05T07:34:02.027365+0000","last_unstale":"2024-02-05T07:34:39.891661+0000","last_undegraded":"2024-02-05T07:34:39.891661+0000","last_fullsized":"2024-02-05T07:34:39.891661+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:53:47.907255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891621+0000","last_change":"2024-02-05T07:33:58.001194+0000","last_active":"2024-02-05T07:34:39.891621+0000","last_peered":"2024-02-05T07:34:39.891621+0000","last_clean":"2024-02-05T07:34:39.891621+0000","last_became_active":"2024-02-05T07:33:58.000798+0000","last_became_peered":"2024-02-05T07:33:58.000798+0000","last_unstale":"2024-02-05T07:34:39.891621+0000","last_undegraded":"2024-02-05T07:34:39.891621+0000","last_fullsized":"2024-02-05T07:34:39.891621+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:41:33.103435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871781+0000","last_change":"2024-02-05T07:34:00.017876+0000","last_active":"2024-02-05T07:34:39.871781+0000","last_peered":"2024-02-05T07:34:39.871781+0000","last_clean":"2024-02-05T07:34:39.871781+0000","last_became_active":"2024-02-05T07:34:00.017471+0000","last_became_peered":"2024-02-05T07:34:00.017471+0000","last_unstale":"2024-02-05T07:34:39.871781+0000","last_undegraded":"2024-02-05T07:34:39.871781+0000","last_fullsized":"2024-02-05T07:34:39.871781+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:13:28.544420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"70'1","reported_seq":22,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735186+0000","last_change":"2024-02-05T07:34:04.035628+0000","last_active":"2024-02-05T07:34:39.735186+0000","last_peered":"2024-02-05T07:34:39.735186+0000","last_clean":"2024-02-05T07:34:39.735186+0000","last_became_active":"2024-02-05T07:34:04.035317+0000","last_became_peered":"2024-02-05T07:34:04.035317+0000","last_unstale":"2024-02-05T07:34:39.735186+0000","last_undegraded":"2024-02-05T07:34:39.735186+0000","last_fullsized":"2024-02-05T07:34:39.735186+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:14:58.016909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"70'8","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060911+0000","last_change":"2024-02-05T07:34:02.023399+0000","last_active":"2024-02-05T07:35:13.060911+0000","last_peered":"2024-02-05T07:35:13.060911+0000","last_clean":"2024-02-05T07:35:13.060911+0000","last_became_active":"2024-02-05T07:34:02.023100+0000","last_became_peered":"2024-02-05T07:34:02.023100+0000","last_unstale":"2024-02-05T07:35:13.060911+0000","last_undegraded":"2024-02-05T07:35:13.060911+0000","last_fullsized":"2024-02-05T07:35:13.060911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:56:16.025653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619268+0000","last_change":"2024-02-05T07:33:58.000793+0000","last_active":"2024-02-05T07:34:39.619268+0000","last_peered":"2024-02-05T07:34:39.619268+0000","last_clean":"2024-02-05T07:34:39.619268+0000","last_became_active":"2024-02-05T07:33:58.000509+0000","last_became_peered":"2024-02-05T07:33:58.000509+0000","last_unstale":"2024-02-05T07:34:39.619268+0000","last_undegraded":"2024-02-05T07:34:39.619268+0000","last_fullsized":"2024-02-05T07:34:39.619268+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:22:07.034225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462730+0000","last_change":"2024-02-05T07:34:00.268609+0000","last_active":"2024-02-05T07:34:40.462730+0000","last_peered":"2024-02-05T07:34:40.462730+0000","last_clean":"2024-02-05T07:34:40.462730+0000","last_became_active":"2024-02-05T07:34:00.268317+0000","last_became_peered":"2024-02-05T07:34:00.268317+0000","last_unstale":"2024-02-05T07:34:40.462730+0000","last_undegraded":"2024-02-05T07:34:40.462730+0000","last_fullsized":"2024-02-05T07:34:40.462730+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:51:42.501493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619866+0000","last_change":"2024-02-05T07:34:04.040699+0000","last_active":"2024-02-05T07:34:39.619866+0000","last_peered":"2024-02-05T07:34:39.619866+0000","last_clean":"2024-02-05T07:34:39.619866+0000","last_became_active":"2024-02-05T07:34:04.036768+0000","last_became_peered":"2024-02-05T07:34:04.036768+0000","last_unstale":"2024-02-05T07:34:39.619866+0000","last_undegraded":"2024-02-05T07:34:39.619866+0000","last_fullsized":"2024-02-05T07:34:39.619866+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:16:50.721705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463557+0000","last_change":"2024-02-05T07:34:02.029568+0000","last_active":"2024-02-05T07:34:40.463557+0000","last_peered":"2024-02-05T07:34:40.463557+0000","last_clean":"2024-02-05T07:34:40.463557+0000","last_became_active":"2024-02-05T07:34:02.028050+0000","last_became_peered":"2024-02-05T07:34:02.028050+0000","last_unstale":"2024-02-05T07:34:40.463557+0000","last_undegraded":"2024-02-05T07:34:40.463557+0000","last_fullsized":"2024-02-05T07:34:40.463557+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:48:23.146415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870556+0000","last_change":"2024-02-05T07:33:57.999919+0000","last_active":"2024-02-05T07:34:39.870556+0000","last_peered":"2024-02-05T07:34:39.870556+0000","last_clean":"2024-02-05T07:34:39.870556+0000","last_became_active":"2024-02-05T07:33:57.999622+0000","last_became_peered":"2024-02-05T07:33:57.999622+0000","last_unstale":"2024-02-05T07:34:39.870556+0000","last_undegraded":"2024-02-05T07:34:39.870556+0000","last_fullsized":"2024-02-05T07:34:39.870556+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:55:35.622235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.618911+0000","last_change":"2024-02-05T07:34:00.017655+0000","last_active":"2024-02-05T07:34:39.618911+0000","last_peered":"2024-02-05T07:34:39.618911+0000","last_clean":"2024-02-05T07:34:39.618911+0000","last_became_active":"2024-02-05T07:34:00.017281+0000","last_became_peered":"2024-02-05T07:34:00.017281+0000","last_unstale":"2024-02-05T07:34:39.618911+0000","last_undegraded":"2024-02-05T07:34:39.618911+0000","last_fullsized":"2024-02-05T07:34:39.618911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:39:04.393070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734765+0000","last_change":"2024-02-05T07:34:04.035415+0000","last_active":"2024-02-05T07:34:39.734765+0000","last_peered":"2024-02-05T07:34:39.734765+0000","last_clean":"2024-02-05T07:34:39.734765+0000","last_became_active":"2024-02-05T07:34:04.035027+0000","last_became_peered":"2024-02-05T07:34:04.035027+0000","last_unstale":"2024-02-05T07:34:39.734765+0000","last_undegraded":"2024-02-05T07:34:39.734765+0000","last_fullsized":"2024-02-05T07:34:39.734765+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:11:05.242662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.618255+0000","last_change":"2024-02-05T07:34:02.028707+0000","last_active":"2024-02-05T07:34:39.618255+0000","last_peered":"2024-02-05T07:34:39.618255+0000","last_clean":"2024-02-05T07:34:39.618255+0000","last_became_active":"2024-02-05T07:34:02.028012+0000","last_became_peered":"2024-02-05T07:34:02.028012+0000","last_unstale":"2024-02-05T07:34:39.618255+0000","last_undegraded":"2024-02-05T07:34:39.618255+0000","last_fullsized":"2024-02-05T07:34:39.618255+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:23:57.296432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463515+0000","last_change":"2024-02-05T07:33:58.006055+0000","last_active":"2024-02-05T07:34:40.463515+0000","last_peered":"2024-02-05T07:34:40.463515+0000","last_clean":"2024-02-05T07:34:40.463515+0000","last_became_active":"2024-02-05T07:33:58.004979+0000","last_became_peered":"2024-02-05T07:33:58.004979+0000","last_unstale":"2024-02-05T07:34:40.463515+0000","last_undegraded":"2024-02-05T07:34:40.463515+0000","last_fullsized":"2024-02-05T07:34:40.463515+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:47:46.304781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"70'4","reported_seq":35,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870628+0000","last_change":"2024-02-05T07:34:00.017835+0000","last_active":"2024-02-05T07:34:39.870628+0000","last_peered":"2024-02-05T07:34:39.870628+0000","last_clean":"2024-02-05T07:34:39.870628+0000","last_became_active":"2024-02-05T07:34:00.017151+0000","last_became_peered":"2024-02-05T07:34:00.017151+0000","last_unstale":"2024-02-05T07:34:39.870628+0000","last_undegraded":"2024-02-05T07:34:39.870628+0000","last_fullsized":"2024-02-05T07:34:39.870628+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:35:31.486048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870933+0000","last_change":"2024-02-05T07:34:04.367657+0000","last_active":"2024-02-05T07:34:39.870933+0000","last_peered":"2024-02-05T07:34:39.870933+0000","last_clean":"2024-02-05T07:34:39.870933+0000","last_became_active":"2024-02-05T07:34:04.367251+0000","last_became_peered":"2024-02-05T07:34:04.367251+0000","last_unstale":"2024-02-05T07:34:39.870933+0000","last_undegraded":"2024-02-05T07:34:39.870933+0000","last_fullsized":"2024-02-05T07:34:39.870933+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:32:24.063406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734985+0000","last_change":"2024-02-05T07:34:02.030849+0000","last_active":"2024-02-05T07:34:39.734985+0000","last_peered":"2024-02-05T07:34:39.734985+0000","last_clean":"2024-02-05T07:34:39.734985+0000","last_became_active":"2024-02-05T07:34:02.030572+0000","last_became_peered":"2024-02-05T07:34:02.030572+0000","last_unstale":"2024-02-05T07:34:39.734985+0000","last_undegraded":"2024-02-05T07:34:39.734985+0000","last_fullsized":"2024-02-05T07:34:39.734985+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:54:30.418129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724202+0000","last_change":"2024-02-05T07:33:58.001589+0000","last_active":"2024-02-05T07:34:39.724202+0000","last_peered":"2024-02-05T07:34:39.724202+0000","last_clean":"2024-02-05T07:34:39.724202+0000","last_became_active":"2024-02-05T07:33:58.001239+0000","last_became_peered":"2024-02-05T07:33:58.001239+0000","last_unstale":"2024-02-05T07:34:39.724202+0000","last_undegraded":"2024-02-05T07:34:39.724202+0000","last_fullsized":"2024-02-05T07:34:39.724202+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:00:58.400180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.872013+0000","last_change":"2024-02-05T07:34:00.267061+0000","last_active":"2024-02-05T07:34:39.872013+0000","last_peered":"2024-02-05T07:34:39.872013+0000","last_clean":"2024-02-05T07:34:39.872013+0000","last_became_active":"2024-02-05T07:34:00.266778+0000","last_became_peered":"2024-02-05T07:34:00.266778+0000","last_unstale":"2024-02-05T07:34:39.872013+0000","last_undegraded":"2024-02-05T07:34:39.872013+0000","last_fullsized":"2024-02-05T07:34:39.872013+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:44:46.908614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462618+0000","last_change":"2024-02-05T07:34:04.041064+0000","last_active":"2024-02-05T07:34:40.462618+0000","last_peered":"2024-02-05T07:34:40.462618+0000","last_clean":"2024-02-05T07:34:40.462618+0000","last_became_active":"2024-02-05T07:34:04.040740+0000","last_became_peered":"2024-02-05T07:34:04.040740+0000","last_unstale":"2024-02-05T07:34:40.462618+0000","last_undegraded":"2024-02-05T07:34:40.462618+0000","last_fullsized":"2024-02-05T07:34:40.462618+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:42:54.475455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871664+0000","last_change":"2024-02-05T07:34:02.024868+0000","last_active":"2024-02-05T07:34:39.871664+0000","last_peered":"2024-02-05T07:34:39.871664+0000","last_clean":"2024-02-05T07:34:39.871664+0000","last_became_active":"2024-02-05T07:34:02.024582+0000","last_became_peered":"2024-02-05T07:34:02.024582+0000","last_unstale":"2024-02-05T07:34:39.871664+0000","last_undegraded":"2024-02-05T07:34:39.871664+0000","last_fullsized":"2024-02-05T07:34:39.871664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:22:11.932379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735275+0000","last_change":"2024-02-05T07:33:58.002496+0000","last_active":"2024-02-05T07:34:39.735275+0000","last_peered":"2024-02-05T07:34:39.735275+0000","last_clean":"2024-02-05T07:34:39.735275+0000","last_became_active":"2024-02-05T07:33:58.002039+0000","last_became_peered":"2024-02-05T07:33:58.002039+0000","last_unstale":"2024-02-05T07:34:39.735275+0000","last_undegraded":"2024-02-05T07:34:39.735275+0000","last_fullsized":"2024-02-05T07:34:39.735275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:11:20.328274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":23,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723568+0000","last_change":"2024-02-05T07:34:16.962178+0000","last_active":"2024-02-05T07:34:39.723568+0000","last_peered":"2024-02-05T07:34:39.723568+0000","last_clean":"2024-02-05T07:34:39.723568+0000","last_became_active":"2024-02-05T07:34:16.954846+0000","last_became_peered":"2024-02-05T07:34:16.954846+0000","last_unstale":"2024-02-05T07:34:39.723568+0000","last_undegraded":"2024-02-05T07:34:39.723568+0000","last_fullsized":"2024-02-05T07:34:39.723568+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:33:16.266150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870844+0000","last_change":"2024-02-05T07:34:04.037934+0000","last_active":"2024-02-05T07:34:39.870844+0000","last_peered":"2024-02-05T07:34:39.870844+0000","last_clean":"2024-02-05T07:34:39.870844+0000","last_became_active":"2024-02-05T07:34:04.037562+0000","last_became_peered":"2024-02-05T07:34:04.037562+0000","last_unstale":"2024-02-05T07:34:39.870844+0000","last_undegraded":"2024-02-05T07:34:39.870844+0000","last_fullsized":"2024-02-05T07:34:39.870844+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:45:16.032769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620350+0000","last_change":"2024-02-05T07:34:02.025940+0000","last_active":"2024-02-05T07:34:39.620350+0000","last_peered":"2024-02-05T07:34:39.620350+0000","last_clean":"2024-02-05T07:34:39.620350+0000","last_became_active":"2024-02-05T07:34:02.025602+0000","last_became_peered":"2024-02-05T07:34:02.025602+0000","last_unstale":"2024-02-05T07:34:39.620350+0000","last_undegraded":"2024-02-05T07:34:39.620350+0000","last_fullsized":"2024-02-05T07:34:39.620350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:34:03.050275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891913+0000","last_change":"2024-02-05T07:33:58.004045+0000","last_active":"2024-02-05T07:34:39.891913+0000","last_peered":"2024-02-05T07:34:39.891913+0000","last_clean":"2024-02-05T07:34:39.891913+0000","last_became_active":"2024-02-05T07:33:58.003717+0000","last_became_peered":"2024-02-05T07:33:58.003717+0000","last_unstale":"2024-02-05T07:34:39.891913+0000","last_undegraded":"2024-02-05T07:34:39.891913+0000","last_fullsized":"2024-02-05T07:34:39.891913+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:59:10.423222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891696+0000","last_change":"2024-02-05T07:34:00.014998+0000","last_active":"2024-02-05T07:34:39.891696+0000","last_peered":"2024-02-05T07:34:39.891696+0000","last_clean":"2024-02-05T07:34:39.891696+0000","last_became_active":"2024-02-05T07:34:00.014647+0000","last_became_peered":"2024-02-05T07:34:00.014647+0000","last_unstale":"2024-02-05T07:34:39.891696+0000","last_undegraded":"2024-02-05T07:34:39.891696+0000","last_fullsized":"2024-02-05T07:34:39.891696+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:47:53.135296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463458+0000","last_change":"2024-02-05T07:34:04.039256+0000","last_active":"2024-02-05T07:34:40.463458+0000","last_peered":"2024-02-05T07:34:40.463458+0000","last_clean":"2024-02-05T07:34:40.463458+0000","last_became_active":"2024-02-05T07:34:04.038858+0000","last_became_peered":"2024-02-05T07:34:04.038858+0000","last_unstale":"2024-02-05T07:34:40.463458+0000","last_undegraded":"2024-02-05T07:34:40.463458+0000","last_fullsized":"2024-02-05T07:34:40.463458+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:14:50.172470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"70'8","reported_seq":53,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060930+0000","last_change":"2024-02-05T07:34:02.032169+0000","last_active":"2024-02-05T07:35:13.060930+0000","last_peered":"2024-02-05T07:35:13.060930+0000","last_clean":"2024-02-05T07:35:13.060930+0000","last_became_active":"2024-02-05T07:34:02.031873+0000","last_became_peered":"2024-02-05T07:34:02.031873+0000","last_unstale":"2024-02-05T07:35:13.060930+0000","last_undegraded":"2024-02-05T07:35:13.060930+0000","last_fullsized":"2024-02-05T07:35:13.060930+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:25:37.637027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734872+0000","last_change":"2024-02-05T07:33:58.001784+0000","last_active":"2024-02-05T07:34:39.734872+0000","last_peered":"2024-02-05T07:34:39.734872+0000","last_clean":"2024-02-05T07:34:39.734872+0000","last_became_active":"2024-02-05T07:33:58.001426+0000","last_became_peered":"2024-02-05T07:33:58.001426+0000","last_unstale":"2024-02-05T07:34:39.734872+0000","last_undegraded":"2024-02-05T07:34:39.734872+0000","last_fullsized":"2024-02-05T07:34:39.734872+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:17:43.462815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"70'10","reported_seq":62,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871896+0000","last_change":"2024-02-05T07:34:16.968302+0000","last_active":"2024-02-05T07:34:39.871896+0000","last_peered":"2024-02-05T07:34:39.871896+0000","last_clean":"2024-02-05T07:34:39.871896+0000","last_became_active":"2024-02-05T07:34:16.955303+0000","last_became_peered":"2024-02-05T07:34:16.955303+0000","last_unstale":"2024-02-05T07:34:39.871896+0000","last_undegraded":"2024-02-05T07:34:39.871896+0000","last_fullsized":"2024-02-05T07:34:39.871896+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:52:03.552702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892333+0000","last_change":"2024-02-05T07:34:04.038358+0000","last_active":"2024-02-05T07:34:39.892333+0000","last_peered":"2024-02-05T07:34:39.892333+0000","last_clean":"2024-02-05T07:34:39.892333+0000","last_became_active":"2024-02-05T07:34:04.037736+0000","last_became_peered":"2024-02-05T07:34:04.037736+0000","last_unstale":"2024-02-05T07:34:39.892333+0000","last_undegraded":"2024-02-05T07:34:39.892333+0000","last_fullsized":"2024-02-05T07:34:39.892333+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:58:26.728709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060803+0000","last_change":"2024-02-05T07:34:02.024289+0000","last_active":"2024-02-05T07:35:13.060803+0000","last_peered":"2024-02-05T07:35:13.060803+0000","last_clean":"2024-02-05T07:35:13.060803+0000","last_became_active":"2024-02-05T07:34:02.023960+0000","last_became_peered":"2024-02-05T07:34:02.023960+0000","last_unstale":"2024-02-05T07:35:13.060803+0000","last_undegraded":"2024-02-05T07:35:13.060803+0000","last_fullsized":"2024-02-05T07:35:13.060803+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:03:01.435915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"63'1","reported_seq":39,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463110+0000","last_change":"2024-02-05T07:33:57.996408+0000","last_active":"2024-02-05T07:34:40.463110+0000","last_peered":"2024-02-05T07:34:40.463110+0000","last_clean":"2024-02-05T07:34:40.463110+0000","last_became_active":"2024-02-05T07:33:57.996091+0000","last_became_peered":"2024-02-05T07:33:57.996091+0000","last_unstale":"2024-02-05T07:34:40.463110+0000","last_undegraded":"2024-02-05T07:34:40.463110+0000","last_fullsized":"2024-02-05T07:34:40.463110+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:34:50.649331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724307+0000","last_change":"2024-02-05T07:34:00.268943+0000","last_active":"2024-02-05T07:34:39.724307+0000","last_peered":"2024-02-05T07:34:39.724307+0000","last_clean":"2024-02-05T07:34:39.724307+0000","last_became_active":"2024-02-05T07:34:00.268603+0000","last_became_peered":"2024-02-05T07:34:00.268603+0000","last_unstale":"2024-02-05T07:34:39.724307+0000","last_undegraded":"2024-02-05T07:34:39.724307+0000","last_fullsized":"2024-02-05T07:34:39.724307+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:01:20.645935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617859+0000","last_change":"2024-02-05T07:34:04.033809+0000","last_active":"2024-02-05T07:34:39.617859+0000","last_peered":"2024-02-05T07:34:39.617859+0000","last_clean":"2024-02-05T07:34:39.617859+0000","last_became_active":"2024-02-05T07:34:04.033503+0000","last_became_peered":"2024-02-05T07:34:04.033503+0000","last_unstale":"2024-02-05T07:34:39.617859+0000","last_undegraded":"2024-02-05T07:34:39.617859+0000","last_fullsized":"2024-02-05T07:34:39.617859+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:11:59.491170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871477+0000","last_change":"2024-02-05T07:34:02.025617+0000","last_active":"2024-02-05T07:34:39.871477+0000","last_peered":"2024-02-05T07:34:39.871477+0000","last_clean":"2024-02-05T07:34:39.871477+0000","last_became_active":"2024-02-05T07:34:02.025325+0000","last_became_peered":"2024-02-05T07:34:02.025325+0000","last_unstale":"2024-02-05T07:34:39.871477+0000","last_undegraded":"2024-02-05T07:34:39.871477+0000","last_fullsized":"2024-02-05T07:34:39.871477+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:48:15.389657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734755+0000","last_change":"2024-02-05T07:33:57.998300+0000","last_active":"2024-02-05T07:34:39.734755+0000","last_peered":"2024-02-05T07:34:39.734755+0000","last_clean":"2024-02-05T07:34:39.734755+0000","last_became_active":"2024-02-05T07:33:57.998008+0000","last_became_peered":"2024-02-05T07:33:57.998008+0000","last_unstale":"2024-02-05T07:34:39.734755+0000","last_undegraded":"2024-02-05T07:34:39.734755+0000","last_fullsized":"2024-02-05T07:34:39.734755+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:41:35.493577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462569+0000","last_change":"2024-02-05T07:34:00.269352+0000","last_active":"2024-02-05T07:34:40.462569+0000","last_peered":"2024-02-05T07:34:40.462569+0000","last_clean":"2024-02-05T07:34:40.462569+0000","last_became_active":"2024-02-05T07:34:00.269046+0000","last_became_peered":"2024-02-05T07:34:00.269046+0000","last_unstale":"2024-02-05T07:34:40.462569+0000","last_undegraded":"2024-02-05T07:34:40.462569+0000","last_fullsized":"2024-02-05T07:34:40.462569+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:08:19.123451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735264+0000","last_change":"2024-02-05T07:34:04.369274+0000","last_active":"2024-02-05T07:34:39.735264+0000","last_peered":"2024-02-05T07:34:39.735264+0000","last_clean":"2024-02-05T07:34:39.735264+0000","last_became_active":"2024-02-05T07:34:04.368916+0000","last_became_peered":"2024-02-05T07:34:04.368916+0000","last_unstale":"2024-02-05T07:34:39.735264+0000","last_undegraded":"2024-02-05T07:34:39.735264+0000","last_fullsized":"2024-02-05T07:34:39.735264+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:56:29.979066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724292+0000","last_change":"2024-02-05T07:34:02.023606+0000","last_active":"2024-02-05T07:34:39.724292+0000","last_peered":"2024-02-05T07:34:39.724292+0000","last_clean":"2024-02-05T07:34:39.724292+0000","last_became_active":"2024-02-05T07:34:02.023313+0000","last_became_peered":"2024-02-05T07:34:02.023313+0000","last_unstale":"2024-02-05T07:34:39.724292+0000","last_undegraded":"2024-02-05T07:34:39.724292+0000","last_fullsized":"2024-02-05T07:34:39.724292+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:59:45.136815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.618209+0000","last_change":"2024-02-05T07:33:57.996598+0000","last_active":"2024-02-05T07:34:39.618209+0000","last_peered":"2024-02-05T07:34:39.618209+0000","last_clean":"2024-02-05T07:34:39.618209+0000","last_became_active":"2024-02-05T07:33:57.996279+0000","last_became_peered":"2024-02-05T07:33:57.996279+0000","last_unstale":"2024-02-05T07:34:39.618209+0000","last_undegraded":"2024-02-05T07:34:39.618209+0000","last_fullsized":"2024-02-05T07:34:39.618209+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:10:55.931975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'6","reported_seq":38,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871670+0000","last_change":"2024-02-05T07:34:00.268196+0000","last_active":"2024-02-05T07:34:39.871670+0000","last_peered":"2024-02-05T07:34:39.871670+0000","last_clean":"2024-02-05T07:34:39.871670+0000","last_became_active":"2024-02-05T07:34:00.267853+0000","last_became_peered":"2024-02-05T07:34:00.267853+0000","last_unstale":"2024-02-05T07:34:39.871670+0000","last_undegraded":"2024-02-05T07:34:39.871670+0000","last_fullsized":"2024-02-05T07:34:39.871670+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:28:54.227264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723845+0000","last_change":"2024-02-05T07:34:04.367690+0000","last_active":"2024-02-05T07:34:39.723845+0000","last_peered":"2024-02-05T07:34:39.723845+0000","last_clean":"2024-02-05T07:34:39.723845+0000","last_became_active":"2024-02-05T07:34:04.367306+0000","last_became_peered":"2024-02-05T07:34:04.367306+0000","last_unstale":"2024-02-05T07:34:39.723845+0000","last_undegraded":"2024-02-05T07:34:39.723845+0000","last_fullsized":"2024-02-05T07:34:39.723845+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:55:11.452215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"63'3","reported_seq":51,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735555+0000","last_change":"2024-02-05T07:33:58.004760+0000","last_active":"2024-02-05T07:34:39.735555+0000","last_peered":"2024-02-05T07:34:39.735555+0000","last_clean":"2024-02-05T07:34:39.735555+0000","last_became_active":"2024-02-05T07:33:58.003077+0000","last_became_peered":"2024-02-05T07:33:58.003077+0000","last_unstale":"2024-02-05T07:34:39.735555+0000","last_undegraded":"2024-02-05T07:34:39.735555+0000","last_fullsized":"2024-02-05T07:34:39.735555+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:41:41.829590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":490,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723572+0000","last_change":"2024-02-05T07:34:00.269682+0000","last_active":"2024-02-05T07:34:39.723572+0000","last_peered":"2024-02-05T07:34:39.723572+0000","last_clean":"2024-02-05T07:34:39.723572+0000","last_became_active":"2024-02-05T07:34:00.269318+0000","last_became_peered":"2024-02-05T07:34:00.269318+0000","last_unstale":"2024-02-05T07:34:39.723572+0000","last_undegraded":"2024-02-05T07:34:39.723572+0000","last_fullsized":"2024-02-05T07:34:39.723572+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:34:01.826854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060929+0000","last_change":"2024-02-05T07:34:02.026069+0000","last_active":"2024-02-05T07:35:13.060929+0000","last_peered":"2024-02-05T07:35:13.060929+0000","last_clean":"2024-02-05T07:35:13.060929+0000","last_became_active":"2024-02-05T07:34:02.025793+0000","last_became_peered":"2024-02-05T07:34:02.025793+0000","last_unstale":"2024-02-05T07:35:13.060929+0000","last_undegraded":"2024-02-05T07:35:13.060929+0000","last_fullsized":"2024-02-05T07:35:13.060929+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:49:25.811278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871575+0000","last_change":"2024-02-05T07:34:04.040764+0000","last_active":"2024-02-05T07:34:39.871575+0000","last_peered":"2024-02-05T07:34:39.871575+0000","last_clean":"2024-02-05T07:34:39.871575+0000","last_became_active":"2024-02-05T07:34:04.040487+0000","last_became_peered":"2024-02-05T07:34:04.040487+0000","last_unstale":"2024-02-05T07:34:39.871575+0000","last_undegraded":"2024-02-05T07:34:39.871575+0000","last_fullsized":"2024-02-05T07:34:39.871575+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:36:37.626200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735426+0000","last_change":"2024-02-05T07:33:58.005760+0000","last_active":"2024-02-05T07:34:39.735426+0000","last_peered":"2024-02-05T07:34:39.735426+0000","last_clean":"2024-02-05T07:34:39.735426+0000","last_became_active":"2024-02-05T07:33:58.005415+0000","last_became_peered":"2024-02-05T07:33:58.005415+0000","last_unstale":"2024-02-05T07:34:39.735426+0000","last_undegraded":"2024-02-05T07:34:39.735426+0000","last_fullsized":"2024-02-05T07:34:39.735426+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:18:46.531096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"70'12","reported_seq":52,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891668+0000","last_change":"2024-02-05T07:34:00.016626+0000","last_active":"2024-02-05T07:34:39.891668+0000","last_peered":"2024-02-05T07:34:39.891668+0000","last_clean":"2024-02-05T07:34:39.891668+0000","last_became_active":"2024-02-05T07:34:00.016271+0000","last_became_peered":"2024-02-05T07:34:00.016271+0000","last_unstale":"2024-02-05T07:34:39.891668+0000","last_undegraded":"2024-02-05T07:34:39.891668+0000","last_fullsized":"2024-02-05T07:34:39.891668+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:13:18.688651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.616782+0000","last_change":"2024-02-05T07:34:02.023556+0000","last_active":"2024-02-05T07:34:39.616782+0000","last_peered":"2024-02-05T07:34:39.616782+0000","last_clean":"2024-02-05T07:34:39.616782+0000","last_became_active":"2024-02-05T07:34:02.023223+0000","last_became_peered":"2024-02-05T07:34:02.023223+0000","last_unstale":"2024-02-05T07:34:39.616782+0000","last_undegraded":"2024-02-05T07:34:39.616782+0000","last_fullsized":"2024-02-05T07:34:39.616782+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:35:48.616191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.734591+0000","last_change":"2024-02-05T07:34:04.038558+0000","last_active":"2024-02-05T07:34:39.734591+0000","last_peered":"2024-02-05T07:34:39.734591+0000","last_clean":"2024-02-05T07:34:39.734591+0000","last_became_active":"2024-02-05T07:34:04.038246+0000","last_became_peered":"2024-02-05T07:34:04.038246+0000","last_unstale":"2024-02-05T07:34:39.734591+0000","last_undegraded":"2024-02-05T07:34:39.734591+0000","last_fullsized":"2024-02-05T07:34:39.734591+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:43:36.952113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617324+0000","last_change":"2024-02-05T07:34:02.024571+0000","last_active":"2024-02-05T07:34:39.617324+0000","last_peered":"2024-02-05T07:34:39.617324+0000","last_clean":"2024-02-05T07:34:39.617324+0000","last_became_active":"2024-02-05T07:34:02.024254+0000","last_became_peered":"2024-02-05T07:34:02.024254+0000","last_unstale":"2024-02-05T07:34:39.617324+0000","last_undegraded":"2024-02-05T07:34:39.617324+0000","last_fullsized":"2024-02-05T07:34:39.617324+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:43:35.066546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735492+0000","last_change":"2024-02-05T07:33:58.000325+0000","last_active":"2024-02-05T07:34:39.735492+0000","last_peered":"2024-02-05T07:34:39.735492+0000","last_clean":"2024-02-05T07:34:39.735492+0000","last_became_active":"2024-02-05T07:33:57.997371+0000","last_became_peered":"2024-02-05T07:33:57.997371+0000","last_unstale":"2024-02-05T07:34:39.735492+0000","last_undegraded":"2024-02-05T07:34:39.735492+0000","last_fullsized":"2024-02-05T07:34:39.735492+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:07:27.092717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"70'19","reported_seq":60,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619625+0000","last_change":"2024-02-05T07:34:00.267846+0000","last_active":"2024-02-05T07:34:39.619625+0000","last_peered":"2024-02-05T07:34:39.619625+0000","last_clean":"2024-02-05T07:34:39.619625+0000","last_became_active":"2024-02-05T07:34:00.267455+0000","last_became_peered":"2024-02-05T07:34:00.267455+0000","last_unstale":"2024-02-05T07:34:39.619625+0000","last_undegraded":"2024-02-05T07:34:39.619625+0000","last_fullsized":"2024-02-05T07:34:39.619625+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:57:53.314787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462722+0000","last_change":"2024-02-05T07:34:04.029044+0000","last_active":"2024-02-05T07:34:40.462722+0000","last_peered":"2024-02-05T07:34:40.462722+0000","last_clean":"2024-02-05T07:34:40.462722+0000","last_became_active":"2024-02-05T07:34:04.028683+0000","last_became_peered":"2024-02-05T07:34:04.028683+0000","last_unstale":"2024-02-05T07:34:40.462722+0000","last_undegraded":"2024-02-05T07:34:40.462722+0000","last_fullsized":"2024-02-05T07:34:40.462722+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:42:00.804637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617103+0000","last_change":"2024-02-05T07:34:02.021355+0000","last_active":"2024-02-05T07:34:39.617103+0000","last_peered":"2024-02-05T07:34:39.617103+0000","last_clean":"2024-02-05T07:34:39.617103+0000","last_became_active":"2024-02-05T07:34:02.020996+0000","last_became_peered":"2024-02-05T07:34:02.020996+0000","last_unstale":"2024-02-05T07:34:39.617103+0000","last_undegraded":"2024-02-05T07:34:39.617103+0000","last_fullsized":"2024-02-05T07:34:39.617103+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:43:52.701127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724009+0000","last_change":"2024-02-05T07:33:57.996940+0000","last_active":"2024-02-05T07:34:39.724009+0000","last_peered":"2024-02-05T07:34:39.724009+0000","last_clean":"2024-02-05T07:34:39.724009+0000","last_became_active":"2024-02-05T07:33:57.996646+0000","last_became_peered":"2024-02-05T07:33:57.996646+0000","last_unstale":"2024-02-05T07:34:39.724009+0000","last_undegraded":"2024-02-05T07:34:39.724009+0000","last_fullsized":"2024-02-05T07:34:39.724009+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:30:17.094356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"70'9","reported_seq":45,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462709+0000","last_change":"2024-02-05T07:34:00.020294+0000","last_active":"2024-02-05T07:34:40.462709+0000","last_peered":"2024-02-05T07:34:40.462709+0000","last_clean":"2024-02-05T07:34:40.462709+0000","last_became_active":"2024-02-05T07:34:00.019919+0000","last_became_peered":"2024-02-05T07:34:00.019919+0000","last_unstale":"2024-02-05T07:34:40.462709+0000","last_undegraded":"2024-02-05T07:34:40.462709+0000","last_fullsized":"2024-02-05T07:34:40.462709+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:58:40.646409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871022+0000","last_change":"2024-02-05T07:34:04.037752+0000","last_active":"2024-02-05T07:34:39.871022+0000","last_peered":"2024-02-05T07:34:39.871022+0000","last_clean":"2024-02-05T07:34:39.871022+0000","last_became_active":"2024-02-05T07:34:04.037427+0000","last_became_peered":"2024-02-05T07:34:04.037427+0000","last_unstale":"2024-02-05T07:34:39.871022+0000","last_undegraded":"2024-02-05T07:34:39.871022+0000","last_fullsized":"2024-02-05T07:34:39.871022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:48:18.326146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.723707+0000","last_change":"2024-02-05T07:34:01.010516+0000","last_active":"2024-02-05T07:34:39.723707+0000","last_peered":"2024-02-05T07:34:39.723707+0000","last_clean":"2024-02-05T07:34:39.723707+0000","last_became_active":"2024-02-05T07:33:56.988114+0000","last_became_peered":"2024-02-05T07:33:56.988114+0000","last_unstale":"2024-02-05T07:34:39.723707+0000","last_undegraded":"2024-02-05T07:34:39.723707+0000","last_fullsized":"2024-02-05T07:34:39.723707+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_clean_scrub_stamp":"2024-02-05T07:33:55.974370+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:10:09.273216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060023400000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891588+0000","last_change":"2024-02-05T07:33:57.998412+0000","last_active":"2024-02-05T07:34:39.891588+0000","last_peered":"2024-02-05T07:34:39.891588+0000","last_clean":"2024-02-05T07:34:39.891588+0000","last_became_active":"2024-02-05T07:33:57.998116+0000","last_became_peered":"2024-02-05T07:33:57.998116+0000","last_unstale":"2024-02-05T07:34:39.891588+0000","last_undegraded":"2024-02-05T07:34:39.891588+0000","last_fullsized":"2024-02-05T07:34:39.891588+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:06:00.650203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"70'28","reported_seq":94,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060676+0000","last_change":"2024-02-05T07:34:00.021183+0000","last_active":"2024-02-05T07:35:13.060676+0000","last_peered":"2024-02-05T07:35:13.060676+0000","last_clean":"2024-02-05T07:35:13.060676+0000","last_became_active":"2024-02-05T07:34:00.020716+0000","last_became_peered":"2024-02-05T07:34:00.020716+0000","last_unstale":"2024-02-05T07:35:13.060676+0000","last_undegraded":"2024-02-05T07:35:13.060676+0000","last_fullsized":"2024-02-05T07:35:13.060676+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:48:23.774263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463210+0000","last_change":"2024-02-05T07:34:02.030209+0000","last_active":"2024-02-05T07:34:40.463210+0000","last_peered":"2024-02-05T07:34:40.463210+0000","last_clean":"2024-02-05T07:34:40.463210+0000","last_became_active":"2024-02-05T07:34:02.029888+0000","last_became_peered":"2024-02-05T07:34:02.029888+0000","last_unstale":"2024-02-05T07:34:40.463210+0000","last_undegraded":"2024-02-05T07:34:40.463210+0000","last_fullsized":"2024-02-05T07:34:40.463210+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:33:54.502182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735571+0000","last_change":"2024-02-05T07:34:04.369802+0000","last_active":"2024-02-05T07:34:39.735571+0000","last_peered":"2024-02-05T07:34:39.735571+0000","last_clean":"2024-02-05T07:34:39.735571+0000","last_became_active":"2024-02-05T07:34:04.369516+0000","last_became_peered":"2024-02-05T07:34:04.369516+0000","last_unstale":"2024-02-05T07:34:39.735571+0000","last_undegraded":"2024-02-05T07:34:39.735571+0000","last_fullsized":"2024-02-05T07:34:39.735571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:22:36.531972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619359+0000","last_change":"2024-02-05T07:33:58.001795+0000","last_active":"2024-02-05T07:34:39.619359+0000","last_peered":"2024-02-05T07:34:39.619359+0000","last_clean":"2024-02-05T07:34:39.619359+0000","last_became_active":"2024-02-05T07:33:58.001557+0000","last_became_peered":"2024-02-05T07:33:58.001557+0000","last_unstale":"2024-02-05T07:34:39.619359+0000","last_undegraded":"2024-02-05T07:34:39.619359+0000","last_fullsized":"2024-02-05T07:34:39.619359+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:07:54.743694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.616935+0000","last_change":"2024-02-05T07:34:01.007720+0000","last_active":"2024-02-05T07:34:39.616935+0000","last_peered":"2024-02-05T07:34:39.616935+0000","last_clean":"2024-02-05T07:34:39.616935+0000","last_became_active":"2024-02-05T07:33:56.991598+0000","last_became_peered":"2024-02-05T07:33:56.991598+0000","last_unstale":"2024-02-05T07:34:39.616935+0000","last_undegraded":"2024-02-05T07:34:39.616935+0000","last_fullsized":"2024-02-05T07:34:39.616935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_clean_scrub_stamp":"2024-02-05T07:33:55.974370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:53:20.476350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088865999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"70'13","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.616995+0000","last_change":"2024-02-05T07:34:16.964536+0000","last_active":"2024-02-05T07:34:39.616995+0000","last_peered":"2024-02-05T07:34:39.616995+0000","last_clean":"2024-02-05T07:34:39.616995+0000","last_became_active":"2024-02-05T07:34:16.954625+0000","last_became_peered":"2024-02-05T07:34:16.954625+0000","last_unstale":"2024-02-05T07:34:39.616995+0000","last_undegraded":"2024-02-05T07:34:39.616995+0000","last_fullsized":"2024-02-05T07:34:39.616995+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:35:54.143468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617008+0000","last_change":"2024-02-05T07:34:02.026320+0000","last_active":"2024-02-05T07:34:39.617008+0000","last_peered":"2024-02-05T07:34:39.617008+0000","last_clean":"2024-02-05T07:34:39.617008+0000","last_became_active":"2024-02-05T07:34:02.025101+0000","last_became_peered":"2024-02-05T07:34:02.025101+0000","last_unstale":"2024-02-05T07:34:39.617008+0000","last_undegraded":"2024-02-05T07:34:39.617008+0000","last_fullsized":"2024-02-05T07:34:39.617008+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:26:09.717222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724258+0000","last_change":"2024-02-05T07:34:04.034429+0000","last_active":"2024-02-05T07:34:39.724258+0000","last_peered":"2024-02-05T07:34:39.724258+0000","last_clean":"2024-02-05T07:34:39.724258+0000","last_became_active":"2024-02-05T07:34:04.034113+0000","last_became_peered":"2024-02-05T07:34:04.034113+0000","last_unstale":"2024-02-05T07:34:39.724258+0000","last_undegraded":"2024-02-05T07:34:39.724258+0000","last_fullsized":"2024-02-05T07:34:39.724258+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:55:46.359906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"74'128","reported_seq":278,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:41.718484+0000","last_change":"2024-02-05T07:33:46.958704+0000","last_active":"2024-02-05T07:34:41.718484+0000","last_peered":"2024-02-05T07:34:41.718484+0000","last_clean":"2024-02-05T07:34:41.718484+0000","last_became_active":"2024-02-05T07:33:46.958264+0000","last_became_peered":"2024-02-05T07:33:46.958264+0000","last_unstale":"2024-02-05T07:34:41.718484+0000","last_undegraded":"2024-02-05T07:34:41.718484+0000","last_fullsized":"2024-02-05T07:34:41.718484+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:31:37.941215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:31:37.941215+0000","last_clean_scrub_stamp":"2024-02-05T07:31:37.941215+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:37:04.350788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871119+0000","last_change":"2024-02-05T07:33:57.997187+0000","last_active":"2024-02-05T07:34:39.871119+0000","last_peered":"2024-02-05T07:34:39.871119+0000","last_clean":"2024-02-05T07:34:39.871119+0000","last_became_active":"2024-02-05T07:33:57.996208+0000","last_became_peered":"2024-02-05T07:33:57.996208+0000","last_unstale":"2024-02-05T07:34:39.871119+0000","last_undegraded":"2024-02-05T07:34:39.871119+0000","last_fullsized":"2024-02-05T07:34:39.871119+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:23:39.654459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":68,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.061202+0000","last_change":"2024-02-05T07:34:00.269699+0000","last_active":"2024-02-05T07:35:13.061202+0000","last_peered":"2024-02-05T07:35:13.061202+0000","last_clean":"2024-02-05T07:35:13.061202+0000","last_became_active":"2024-02-05T07:34:00.269325+0000","last_became_peered":"2024-02-05T07:34:00.269325+0000","last_unstale":"2024-02-05T07:35:13.061202+0000","last_undegraded":"2024-02-05T07:35:13.061202+0000","last_fullsized":"2024-02-05T07:35:13.061202+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:50:56.857301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735272+0000","last_change":"2024-02-05T07:34:02.030586+0000","last_active":"2024-02-05T07:34:39.735272+0000","last_peered":"2024-02-05T07:34:39.735272+0000","last_clean":"2024-02-05T07:34:39.735272+0000","last_became_active":"2024-02-05T07:34:02.030193+0000","last_became_peered":"2024-02-05T07:34:02.030193+0000","last_unstale":"2024-02-05T07:34:39.735272+0000","last_undegraded":"2024-02-05T07:34:39.735272+0000","last_fullsized":"2024-02-05T07:34:39.735272+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:44:09.447368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619417+0000","last_change":"2024-02-05T07:34:04.041105+0000","last_active":"2024-02-05T07:34:39.619417+0000","last_peered":"2024-02-05T07:34:39.619417+0000","last_clean":"2024-02-05T07:34:39.619417+0000","last_became_active":"2024-02-05T07:34:04.040764+0000","last_became_peered":"2024-02-05T07:34:04.040764+0000","last_unstale":"2024-02-05T07:34:39.619417+0000","last_undegraded":"2024-02-05T07:34:39.619417+0000","last_fullsized":"2024-02-05T07:34:39.619417+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:11:37.095803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462889+0000","last_change":"2024-02-05T07:33:58.003439+0000","last_active":"2024-02-05T07:34:40.462889+0000","last_peered":"2024-02-05T07:34:40.462889+0000","last_clean":"2024-02-05T07:34:40.462889+0000","last_became_active":"2024-02-05T07:33:58.003130+0000","last_became_peered":"2024-02-05T07:33:58.003130+0000","last_unstale":"2024-02-05T07:34:40.462889+0000","last_undegraded":"2024-02-05T07:34:40.462889+0000","last_fullsized":"2024-02-05T07:34:40.462889+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:48:02.799126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"71'5","reported_seq":121,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:19.588074+0000","last_change":"2024-02-05T07:34:01.011414+0000","last_active":"2024-02-05T07:35:19.588074+0000","last_peered":"2024-02-05T07:35:19.588074+0000","last_clean":"2024-02-05T07:35:19.588074+0000","last_became_active":"2024-02-05T07:33:56.993501+0000","last_became_peered":"2024-02-05T07:33:56.993501+0000","last_unstale":"2024-02-05T07:35:19.588074+0000","last_undegraded":"2024-02-05T07:35:19.588074+0000","last_fullsized":"2024-02-05T07:35:19.588074+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:55.974370+0000","last_clean_scrub_stamp":"2024-02-05T07:33:55.974370+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:24:17.351722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047533500000000002,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":47,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463361+0000","last_change":"2024-02-05T07:34:00.013085+0000","last_active":"2024-02-05T07:34:40.463361+0000","last_peered":"2024-02-05T07:34:40.463361+0000","last_clean":"2024-02-05T07:34:40.463361+0000","last_became_active":"2024-02-05T07:34:00.012744+0000","last_became_peered":"2024-02-05T07:34:00.012744+0000","last_unstale":"2024-02-05T07:34:40.463361+0000","last_undegraded":"2024-02-05T07:34:40.463361+0000","last_fullsized":"2024-02-05T07:34:40.463361+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:05:22.345969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724424+0000","last_change":"2024-02-05T07:34:02.024966+0000","last_active":"2024-02-05T07:34:39.724424+0000","last_peered":"2024-02-05T07:34:39.724424+0000","last_clean":"2024-02-05T07:34:39.724424+0000","last_became_active":"2024-02-05T07:34:02.024451+0000","last_became_peered":"2024-02-05T07:34:02.024451+0000","last_unstale":"2024-02-05T07:34:39.724424+0000","last_undegraded":"2024-02-05T07:34:39.724424+0000","last_fullsized":"2024-02-05T07:34:39.724424+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:20:40.494465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619068+0000","last_change":"2024-02-05T07:34:04.367641+0000","last_active":"2024-02-05T07:34:39.619068+0000","last_peered":"2024-02-05T07:34:39.619068+0000","last_clean":"2024-02-05T07:34:39.619068+0000","last_became_active":"2024-02-05T07:34:04.367313+0000","last_became_peered":"2024-02-05T07:34:04.367313+0000","last_unstale":"2024-02-05T07:34:39.619068+0000","last_undegraded":"2024-02-05T07:34:39.619068+0000","last_fullsized":"2024-02-05T07:34:39.619068+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:31:35.209564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619743+0000","last_change":"2024-02-05T07:33:58.002752+0000","last_active":"2024-02-05T07:34:39.619743+0000","last_peered":"2024-02-05T07:34:39.619743+0000","last_clean":"2024-02-05T07:34:39.619743+0000","last_became_active":"2024-02-05T07:33:58.002457+0000","last_became_peered":"2024-02-05T07:33:58.002457+0000","last_unstale":"2024-02-05T07:34:39.619743+0000","last_undegraded":"2024-02-05T07:34:39.619743+0000","last_fullsized":"2024-02-05T07:34:39.619743+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:05:45.116433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"70'19","reported_seq":65,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462543+0000","last_change":"2024-02-05T07:34:00.267169+0000","last_active":"2024-02-05T07:34:40.462543+0000","last_peered":"2024-02-05T07:34:40.462543+0000","last_clean":"2024-02-05T07:34:40.462543+0000","last_became_active":"2024-02-05T07:34:00.266840+0000","last_became_peered":"2024-02-05T07:34:00.266840+0000","last_unstale":"2024-02-05T07:34:40.462543+0000","last_undegraded":"2024-02-05T07:34:40.462543+0000","last_fullsized":"2024-02-05T07:34:40.462543+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:18:57.927192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619801+0000","last_change":"2024-02-05T07:34:02.032248+0000","last_active":"2024-02-05T07:34:39.619801+0000","last_peered":"2024-02-05T07:34:39.619801+0000","last_clean":"2024-02-05T07:34:39.619801+0000","last_became_active":"2024-02-05T07:34:02.031980+0000","last_became_peered":"2024-02-05T07:34:02.031980+0000","last_unstale":"2024-02-05T07:34:39.619801+0000","last_undegraded":"2024-02-05T07:34:39.619801+0000","last_fullsized":"2024-02-05T07:34:39.619801+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:23:50.868885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.462641+0000","last_change":"2024-02-05T07:34:04.038535+0000","last_active":"2024-02-05T07:34:40.462641+0000","last_peered":"2024-02-05T07:34:40.462641+0000","last_clean":"2024-02-05T07:34:40.462641+0000","last_became_active":"2024-02-05T07:34:04.038217+0000","last_became_peered":"2024-02-05T07:34:04.038217+0000","last_unstale":"2024-02-05T07:34:40.462641+0000","last_undegraded":"2024-02-05T07:34:40.462641+0000","last_fullsized":"2024-02-05T07:34:40.462641+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:52:04.664396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724189+0000","last_change":"2024-02-05T07:33:57.998024+0000","last_active":"2024-02-05T07:34:39.724189+0000","last_peered":"2024-02-05T07:34:39.724189+0000","last_clean":"2024-02-05T07:34:39.724189+0000","last_became_active":"2024-02-05T07:33:57.997657+0000","last_became_peered":"2024-02-05T07:33:57.997657+0000","last_unstale":"2024-02-05T07:34:39.724189+0000","last_undegraded":"2024-02-05T07:34:39.724189+0000","last_fullsized":"2024-02-05T07:34:39.724189+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:14:01.177060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620398+0000","last_change":"2024-02-05T07:34:00.017811+0000","last_active":"2024-02-05T07:34:39.620398+0000","last_peered":"2024-02-05T07:34:39.620398+0000","last_clean":"2024-02-05T07:34:39.620398+0000","last_became_active":"2024-02-05T07:34:00.017422+0000","last_became_peered":"2024-02-05T07:34:00.017422+0000","last_unstale":"2024-02-05T07:34:39.620398+0000","last_undegraded":"2024-02-05T07:34:39.620398+0000","last_fullsized":"2024-02-05T07:34:39.620398+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T17:11:17.303653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060892+0000","last_change":"2024-02-05T07:34:02.029246+0000","last_active":"2024-02-05T07:35:13.060892+0000","last_peered":"2024-02-05T07:35:13.060892+0000","last_clean":"2024-02-05T07:35:13.060892+0000","last_became_active":"2024-02-05T07:34:02.028928+0000","last_became_peered":"2024-02-05T07:34:02.028928+0000","last_unstale":"2024-02-05T07:35:13.060892+0000","last_undegraded":"2024-02-05T07:35:13.060892+0000","last_fullsized":"2024-02-05T07:35:13.060892+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:59:05.616314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.735134+0000","last_change":"2024-02-05T07:34:04.370494+0000","last_active":"2024-02-05T07:34:39.735134+0000","last_peered":"2024-02-05T07:34:39.735134+0000","last_clean":"2024-02-05T07:34:39.735134+0000","last_became_active":"2024-02-05T07:34:04.369638+0000","last_became_peered":"2024-02-05T07:34:04.369638+0000","last_unstale":"2024-02-05T07:34:39.735134+0000","last_undegraded":"2024-02-05T07:34:39.735134+0000","last_fullsized":"2024-02-05T07:34:39.735134+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:09:55.354271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"63'1","reported_seq":34,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:40.463492+0000","last_change":"2024-02-05T07:33:57.999949+0000","last_active":"2024-02-05T07:34:40.463492+0000","last_peered":"2024-02-05T07:34:40.463492+0000","last_clean":"2024-02-05T07:34:40.463492+0000","last_became_active":"2024-02-05T07:33:57.999258+0000","last_became_peered":"2024-02-05T07:33:57.999258+0000","last_unstale":"2024-02-05T07:34:40.463492+0000","last_undegraded":"2024-02-05T07:34:40.463492+0000","last_fullsized":"2024-02-05T07:34:40.463492+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T15:22:07.608338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"70'14","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891620+0000","last_change":"2024-02-05T07:34:00.010028+0000","last_active":"2024-02-05T07:34:39.891620+0000","last_peered":"2024-02-05T07:34:39.891620+0000","last_clean":"2024-02-05T07:34:39.891620+0000","last_became_active":"2024-02-05T07:34:00.009625+0000","last_became_peered":"2024-02-05T07:34:00.009625+0000","last_unstale":"2024-02-05T07:34:39.891620+0000","last_undegraded":"2024-02-05T07:34:39.891620+0000","last_fullsized":"2024-02-05T07:34:39.891620+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:04:05.562130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060754+0000","last_change":"2024-02-05T07:34:02.026329+0000","last_active":"2024-02-05T07:35:13.060754+0000","last_peered":"2024-02-05T07:35:13.060754+0000","last_clean":"2024-02-05T07:35:13.060754+0000","last_became_active":"2024-02-05T07:34:02.026034+0000","last_became_peered":"2024-02-05T07:34:02.026034+0000","last_unstale":"2024-02-05T07:35:13.060754+0000","last_undegraded":"2024-02-05T07:35:13.060754+0000","last_fullsized":"2024-02-05T07:35:13.060754+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T07:46:32.527993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891465+0000","last_change":"2024-02-05T07:34:04.038303+0000","last_active":"2024-02-05T07:34:39.891465+0000","last_peered":"2024-02-05T07:34:39.891465+0000","last_clean":"2024-02-05T07:34:39.891465+0000","last_became_active":"2024-02-05T07:34:04.038019+0000","last_became_peered":"2024-02-05T07:34:04.038019+0000","last_unstale":"2024-02-05T07:34:39.891465+0000","last_undegraded":"2024-02-05T07:34:39.891465+0000","last_fullsized":"2024-02-05T07:34:39.891465+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:15:42.606783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871274+0000","last_change":"2024-02-05T07:33:57.993348+0000","last_active":"2024-02-05T07:34:39.871274+0000","last_peered":"2024-02-05T07:34:39.871274+0000","last_clean":"2024-02-05T07:34:39.871274+0000","last_became_active":"2024-02-05T07:33:57.993051+0000","last_became_peered":"2024-02-05T07:33:57.993051+0000","last_unstale":"2024-02-05T07:34:39.871274+0000","last_undegraded":"2024-02-05T07:34:39.871274+0000","last_fullsized":"2024-02-05T07:34:39.871274+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:26:31.251993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"70'18","reported_seq":61,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870750+0000","last_change":"2024-02-05T07:34:00.266782+0000","last_active":"2024-02-05T07:34:39.870750+0000","last_peered":"2024-02-05T07:34:39.870750+0000","last_clean":"2024-02-05T07:34:39.870750+0000","last_became_active":"2024-02-05T07:34:00.266479+0000","last_became_peered":"2024-02-05T07:34:00.266479+0000","last_unstale":"2024-02-05T07:34:39.870750+0000","last_undegraded":"2024-02-05T07:34:39.870750+0000","last_fullsized":"2024-02-05T07:34:39.870750+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:21:34.761450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891300+0000","last_change":"2024-02-05T07:34:02.027486+0000","last_active":"2024-02-05T07:34:39.891300+0000","last_peered":"2024-02-05T07:34:39.891300+0000","last_clean":"2024-02-05T07:34:39.891300+0000","last_became_active":"2024-02-05T07:34:02.027180+0000","last_became_peered":"2024-02-05T07:34:02.027180+0000","last_unstale":"2024-02-05T07:34:39.891300+0000","last_undegraded":"2024-02-05T07:34:39.891300+0000","last_fullsized":"2024-02-05T07:34:39.891300+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:14:19.617186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724010+0000","last_change":"2024-02-05T07:34:04.033403+0000","last_active":"2024-02-05T07:34:39.724010+0000","last_peered":"2024-02-05T07:34:39.724010+0000","last_clean":"2024-02-05T07:34:39.724010+0000","last_became_active":"2024-02-05T07:34:04.033126+0000","last_became_peered":"2024-02-05T07:34:04.033126+0000","last_unstale":"2024-02-05T07:34:39.724010+0000","last_undegraded":"2024-02-05T07:34:39.724010+0000","last_fullsized":"2024-02-05T07:34:39.724010+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:11:39.907031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060902+0000","last_change":"2024-02-05T07:34:02.027667+0000","last_active":"2024-02-05T07:35:13.060902+0000","last_peered":"2024-02-05T07:35:13.060902+0000","last_clean":"2024-02-05T07:35:13.060902+0000","last_became_active":"2024-02-05T07:34:02.024417+0000","last_became_peered":"2024-02-05T07:34:02.024417+0000","last_unstale":"2024-02-05T07:35:13.060902+0000","last_undegraded":"2024-02-05T07:35:13.060902+0000","last_fullsized":"2024-02-05T07:35:13.060902+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:33:35.007406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.870728+0000","last_change":"2024-02-05T07:33:57.997642+0000","last_active":"2024-02-05T07:34:39.870728+0000","last_peered":"2024-02-05T07:34:39.870728+0000","last_clean":"2024-02-05T07:34:39.870728+0000","last_became_active":"2024-02-05T07:33:57.997361+0000","last_became_peered":"2024-02-05T07:33:57.997361+0000","last_unstale":"2024-02-05T07:34:39.870728+0000","last_undegraded":"2024-02-05T07:34:39.870728+0000","last_fullsized":"2024-02-05T07:34:39.870728+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:32:26.163667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":54,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620140+0000","last_change":"2024-02-05T07:34:00.015169+0000","last_active":"2024-02-05T07:34:39.620140+0000","last_peered":"2024-02-05T07:34:39.620140+0000","last_clean":"2024-02-05T07:34:39.620140+0000","last_became_active":"2024-02-05T07:34:00.014865+0000","last_became_peered":"2024-02-05T07:34:00.014865+0000","last_unstale":"2024-02-05T07:34:39.620140+0000","last_undegraded":"2024-02-05T07:34:39.620140+0000","last_fullsized":"2024-02-05T07:34:39.620140+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:34:39.394930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871216+0000","last_change":"2024-02-05T07:34:04.368042+0000","last_active":"2024-02-05T07:34:39.871216+0000","last_peered":"2024-02-05T07:34:39.871216+0000","last_clean":"2024-02-05T07:34:39.871216+0000","last_became_active":"2024-02-05T07:34:04.367707+0000","last_became_peered":"2024-02-05T07:34:04.367707+0000","last_unstale":"2024-02-05T07:34:39.871216+0000","last_undegraded":"2024-02-05T07:34:39.871216+0000","last_fullsized":"2024-02-05T07:34:39.871216+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:21:16.757195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.724401+0000","last_change":"2024-02-05T07:34:02.024725+0000","last_active":"2024-02-05T07:34:39.724401+0000","last_peered":"2024-02-05T07:34:39.724401+0000","last_clean":"2024-02-05T07:34:39.724401+0000","last_became_active":"2024-02-05T07:34:02.023947+0000","last_became_peered":"2024-02-05T07:34:02.023947+0000","last_unstale":"2024-02-05T07:34:39.724401+0000","last_undegraded":"2024-02-05T07:34:39.724401+0000","last_fullsized":"2024-02-05T07:34:39.724401+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T10:18:17.899108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891394+0000","last_change":"2024-02-05T07:33:58.003615+0000","last_active":"2024-02-05T07:34:39.891394+0000","last_peered":"2024-02-05T07:34:39.891394+0000","last_clean":"2024-02-05T07:34:39.891394+0000","last_became_active":"2024-02-05T07:33:58.003068+0000","last_became_peered":"2024-02-05T07:33:58.003068+0000","last_unstale":"2024-02-05T07:34:39.891394+0000","last_undegraded":"2024-02-05T07:34:39.891394+0000","last_fullsized":"2024-02-05T07:34:39.891394+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:13:46.594789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"70'11","reported_seq":48,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892177+0000","last_change":"2024-02-05T07:34:00.014686+0000","last_active":"2024-02-05T07:34:39.892177+0000","last_peered":"2024-02-05T07:34:39.892177+0000","last_clean":"2024-02-05T07:34:39.892177+0000","last_became_active":"2024-02-05T07:34:00.014313+0000","last_became_peered":"2024-02-05T07:34:00.014313+0000","last_unstale":"2024-02-05T07:34:39.892177+0000","last_undegraded":"2024-02-05T07:34:39.892177+0000","last_fullsized":"2024-02-05T07:34:39.892177+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T18:34:55.487862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.617226+0000","last_change":"2024-02-05T07:34:04.035996+0000","last_active":"2024-02-05T07:34:39.617226+0000","last_peered":"2024-02-05T07:34:39.617226+0000","last_clean":"2024-02-05T07:34:39.617226+0000","last_became_active":"2024-02-05T07:34:04.035256+0000","last_became_peered":"2024-02-05T07:34:04.035256+0000","last_unstale":"2024-02-05T07:34:39.617226+0000","last_undegraded":"2024-02-05T07:34:39.617226+0000","last_fullsized":"2024-02-05T07:34:39.617226+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T19:09:19.804514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.620005+0000","last_change":"2024-02-05T07:34:02.023289+0000","last_active":"2024-02-05T07:34:39.620005+0000","last_peered":"2024-02-05T07:34:39.620005+0000","last_clean":"2024-02-05T07:34:39.620005+0000","last_became_active":"2024-02-05T07:34:02.022920+0000","last_became_peered":"2024-02-05T07:34:02.022920+0000","last_unstale":"2024-02-05T07:34:39.620005+0000","last_undegraded":"2024-02-05T07:34:39.620005+0000","last_fullsized":"2024-02-05T07:34:39.620005+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T16:00:28.904996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.619443+0000","last_change":"2024-02-05T07:33:57.997980+0000","last_active":"2024-02-05T07:34:39.619443+0000","last_peered":"2024-02-05T07:34:39.619443+0000","last_clean":"2024-02-05T07:34:39.619443+0000","last_became_active":"2024-02-05T07:33:57.997673+0000","last_became_peered":"2024-02-05T07:33:57.997673+0000","last_unstale":"2024-02-05T07:34:39.619443+0000","last_undegraded":"2024-02-05T07:34:39.619443+0000","last_fullsized":"2024-02-05T07:34:39.619443+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T14:34:57.570197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":57,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892087+0000","last_change":"2024-02-05T07:34:00.014699+0000","last_active":"2024-02-05T07:34:39.892087+0000","last_peered":"2024-02-05T07:34:39.892087+0000","last_clean":"2024-02-05T07:34:39.892087+0000","last_became_active":"2024-02-05T07:34:00.014140+0000","last_became_peered":"2024-02-05T07:34:00.014140+0000","last_unstale":"2024-02-05T07:34:39.892087+0000","last_undegraded":"2024-02-05T07:34:39.892087+0000","last_fullsized":"2024-02-05T07:34:39.892087+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T09:32:34.659965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.892338+0000","last_change":"2024-02-05T07:34:04.036556+0000","last_active":"2024-02-05T07:34:39.892338+0000","last_peered":"2024-02-05T07:34:39.892338+0000","last_clean":"2024-02-05T07:34:39.892338+0000","last_became_active":"2024-02-05T07:34:04.036254+0000","last_became_peered":"2024-02-05T07:34:04.036254+0000","last_unstale":"2024-02-05T07:34:39.892338+0000","last_undegraded":"2024-02-05T07:34:39.892338+0000","last_fullsized":"2024-02-05T07:34:39.892338+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:03.010310+0000","last_clean_scrub_stamp":"2024-02-05T07:34:03.010310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T13:15:14.107732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"70'8","reported_seq":50,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:35:13.060665+0000","last_change":"2024-02-05T07:34:02.025976+0000","last_active":"2024-02-05T07:35:13.060665+0000","last_peered":"2024-02-05T07:35:13.060665+0000","last_clean":"2024-02-05T07:35:13.060665+0000","last_became_active":"2024-02-05T07:34:02.025623+0000","last_became_peered":"2024-02-05T07:34:02.025623+0000","last_unstale":"2024-02-05T07:35:13.060665+0000","last_undegraded":"2024-02-05T07:35:13.060665+0000","last_fullsized":"2024-02-05T07:35:13.060665+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:34:00.999138+0000","last_clean_scrub_stamp":"2024-02-05T07:34:00.999138+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T08:28:28.422092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.871745+0000","last_change":"2024-02-05T07:33:58.003609+0000","last_active":"2024-02-05T07:34:39.871745+0000","last_peered":"2024-02-05T07:34:39.871745+0000","last_clean":"2024-02-05T07:34:39.871745+0000","last_became_active":"2024-02-05T07:33:58.003173+0000","last_became_peered":"2024-02-05T07:33:58.003173+0000","last_unstale":"2024-02-05T07:34:39.871745+0000","last_undegraded":"2024-02-05T07:34:39.871745+0000","last_fullsized":"2024-02-05T07:34:39.871745+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:56.978513+0000","last_clean_scrub_stamp":"2024-02-05T07:33:56.978513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T11:41:41.688660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":44,"reported_epoch":74,"state":"active+clean","last_fresh":"2024-02-05T07:34:39.891847+0000","last_change":"2024-02-05T07:34:00.016560+0000","last_active":"2024-02-05T07:34:39.891847+0000","last_peered":"2024-02-05T07:34:39.891847+0000","last_clean":"2024-02-05T07:34:39.891847+0000","last_became_active":"2024-02-05T07:34:00.016214+0000","last_became_peered":"2024-02-05T07:34:00.016214+0000","last_unstale":"2024-02-05T07:34:39.891847+0000","last_undegraded":"2024-02-05T07:34:39.891847+0000","last_fullsized":"2024-02-05T07:34:39.891847+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-05T07:33:58.989505+0000","last_clean_scrub_stamp":"2024-02-05T07:33:58.989505+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-06T12:55:28.194806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":697,"num_read_kb":454,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":56,"seq":240518168597,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28408,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716744,"statfs":{"total":95995035648,"available":95965945856,"internally_reserved":0,"allocated":1425408,"data_stored":936896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.52400000000000002}]},{"osd":1,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54700000000000004}]},{"osd":2,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.46200000000000002}]},{"osd":3,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.498}]},{"osd":4,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.65400000000000003}]},{"osd":5,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.437},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.60999999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.59299999999999997}]}]},{"osd":6,"up_from":48,"seq":206158430234,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32544,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712608,"statfs":{"total":95995035648,"available":95961710592,"internally_reserved":0,"allocated":1400832,"data_stored":934586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.53700000000000003,"15min":0.54200000000000004},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.70699999999999996,"5min":0.755,"15min":0.755},"last":0.51100000000000001}]},{"osd":1,"last update":"Mon Feb 5 07:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54300000000000004,"15min":0.54700000000000004},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.72099999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.54500000000000004,"15min":0.54900000000000004},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72899999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Mon Feb 5 07:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.68799999999999994,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.624}]},{"osd":3,"last update":"Mon Feb 5 07:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56200000000000006,"15min":0.56599999999999995},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.79300000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59199999999999997,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.71399999999999997}]},{"osd":4,"last update":"Mon Feb 5 07:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.45300000000000001,"15min":0.44400000000000001},"min":{"1min":0.35399999999999998,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.68100000000000005,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.48699999999999999,"15min":0.48299999999999998},"min":{"1min":0.34200000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.78100000000000003}]},{"osd":5,"last update":"Mon Feb 5 07:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.48899999999999999},"min":{"1min":0.35799999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.78000000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.52800000000000002,"15min":0.52700000000000002},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.77200000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.81499999999999995}]},{"osd":7,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.73899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":851968,"data_stored":345550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.371,"15min":0.33300000000000002},"min":{"1min":0.27100000000000002,"5min":0.157,"15min":0.157},"max":{"1min":0.53400000000000003,"5min":0.628,"15min":0.628},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.375,"15min":0.34399999999999997},"min":{"1min":0.26500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.505,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.40899999999999997}]},{"osd":2,"last update":"Mon Feb 5 07:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.40899999999999997,"15min":0.39900000000000002},"min":{"1min":0.28199999999999997,"5min":0.189,"15min":0.189},"max":{"1min":0.59099999999999997,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.41199999999999998},"min":{"1min":0.27400000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.61199999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.58399999999999996}]},{"osd":3,"last update":"Mon Feb 5 07:35:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.47699999999999998,"15min":0.47499999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45500000000000002,"15min":0.44900000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.45900000000000002}]},{"osd":4,"last update":"Mon Feb 5 07:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.443},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57299999999999995,"15min":0.56899999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.52900000000000003}]},{"osd":5,"last update":"Mon Feb 5 07:34:56 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.65200000000000002,"15min":0.65800000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.65100000000000002}]},{"osd":6,"last update":"Mon Feb 5 07:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63600000000000001,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.79800000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.625},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.621,"15min":0.628},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.80200000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56100000000000005}]},{"osd":7,"last update":"Mon Feb 5 07:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.73599999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28412,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716740,"statfs":{"total":95995035648,"available":95965941760,"internally_reserved":0,"allocated":1429504,"data_stored":936639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 5 07:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.376,"15min":0.34000000000000002},"min":{"1min":0.23599999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.55600000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.42199999999999999,"15min":0.38600000000000001},"min":{"1min":0.307,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.48599999999999999}]},{"osd":2,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40200000000000002,"15min":0.39800000000000002},"min":{"1min":0.28699999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52500000000000002,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.42099999999999999,"15min":0.41899999999999998},"min":{"1min":0.28000000000000003,"5min":0.246,"15min":0.246},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.502}]},{"osd":3,"last update":"Mon Feb 5 07:35:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47599999999999998,"15min":0.48799999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.47899999999999998,"15min":0.501},"min":{"1min":0.27200000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.51800000000000002}]},{"osd":4,"last update":"Mon Feb 5 07:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.625,"15min":0.62},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.61699999999999999}]},{"osd":5,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60499999999999998,"15min":0.60599999999999998},"min":{"1min":0.42699999999999999,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.64100000000000001}]},{"osd":6,"last update":"Mon Feb 5 07:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63400000000000001,"15min":0.64000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.874,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.67600000000000005},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67700000000000005}]},{"osd":7,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.629}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31972,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713180,"statfs":{"total":95995035648,"available":95962296320,"internally_reserved":0,"allocated":811008,"data_stored":344649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.377,"15min":0.36599999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.375},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38600000000000001,"15min":0.378},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.54100000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.29599999999999999}]},{"osd":1,"last update":"Mon Feb 5 07:34:39 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40000000000000002,"15min":0.38800000000000001},"min":{"1min":0.28599999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57799999999999996,"5min":0.621,"15min":0.621},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.42799999999999999,"15min":0.40899999999999997},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.435}]},{"osd":3,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.376,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.39600000000000002}]},{"osd":4,"last update":"Mon Feb 5 07:34:32 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.57599999999999996,"15min":0.56699999999999995},"min":{"1min":0.42699999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.60599999999999998,"15min":0.60299999999999998},"min":{"1min":0.45800000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.53700000000000003}]},{"osd":5,"last update":"Mon Feb 5 07:34:59 2024","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.64000000000000001,"15min":0.64600000000000002},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.81899999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.62,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84799999999999998,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.54700000000000004}]},{"osd":6,"last update":"Mon Feb 5 07:35:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.67200000000000004,"15min":0.68400000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.86099999999999999,"5min":0.998,"15min":0.998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.84299999999999997,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66600000000000004}]},{"osd":7,"last update":"Mon Feb 5 07:34:42 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.629}]}]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32008,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713144,"statfs":{"total":95995035648,"available":95962259456,"internally_reserved":0,"allocated":847872,"data_stored":345983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.45800000000000002,"15min":0.433},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":5.8949999999999996,"5min":5.8949999999999996,"15min":5.8949999999999996},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.437,"15min":0.442},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46899999999999997,"15min":0.47899999999999998},"min":{"1min":0.26500000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.865,"5min":1.865,"15min":1.865},"last":0.374},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45900000000000002,"15min":0.46200000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.1150000000000002,"5min":3.1150000000000002,"15min":3.1150000000000002},"last":0.28199999999999997}]},{"osd":2,"last update":"Mon Feb 5 07:35:07 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.48099999999999998,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.50700000000000001,"15min":0.495},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":5.6529999999999996,"5min":5.6529999999999996,"15min":5.6529999999999996},"last":0.30399999999999999}]},{"osd":4,"last update":"Mon Feb 5 07:34:28 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.60299999999999998,"15min":0.60399999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.60099999999999998,"15min":0.60699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.75600000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.441}]},{"osd":5,"last update":"Mon Feb 5 07:34:53 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.66500000000000004,"15min":0.64000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":5.5979999999999999,"5min":5.5979999999999999,"15min":5.5979999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63600000000000001,"15min":0.63},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.931,"5min":1.931,"15min":1.931},"last":0.49099999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:35:19 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64400000000000002,"15min":0.64500000000000002},"min":{"1min":0.41999999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.823,"5min":1.823,"15min":1.823},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.621,"15min":0.624},"min":{"1min":0.432,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.55100000000000005}]},{"osd":7,"last update":"Mon Feb 5 07:34:45 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":5.6879999999999997,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.52400000000000002}]}]},{"osd":4,"up_from":32,"seq":137438953510,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":345208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55300000000000005,"15min":0.55100000000000005},"min":{"1min":0.376,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74199999999999999,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65500000000000003}]},{"osd":1,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58099999999999996,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.375,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88900000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.59799999999999998}]},{"osd":2,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.76200000000000001,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.52000000000000002}]},{"osd":3,"last update":"Mon Feb 5 07:34:29 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Mon Feb 5 07:34:57 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"min":{"1min":0.28000000000000003,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.65100000000000002,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.433,"5min":0.436,"15min":0.436},"min":{"1min":0.30099999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.61899999999999999,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.42399999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:35:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.29099999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.501,"15min":0.499},"min":{"1min":0.33500000000000002,"5min":0.316,"15min":0.316},"max":{"1min":0.79500000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.40500000000000003}]},{"osd":7,"last update":"Mon Feb 5 07:34:44 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.50700000000000001}]}]},{"osd":5,"up_from":40,"seq":171798691872,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31976,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713176,"statfs":{"total":95995035648,"available":95962292224,"internally_reserved":0,"allocated":815104,"data_stored":344653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66900000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.53600000000000003}]},{"osd":1,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.54700000000000004,"15min":0.55300000000000005},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.72099999999999997,"5min":0.754,"15min":0.754},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.56899999999999995,"15min":0.57699999999999996},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.59899999999999998}]},{"osd":2,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.56699999999999995,"15min":0.57599999999999996},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.57099999999999995}]},{"osd":3,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59699999999999998,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.40500000000000003}]},{"osd":4,"last update":"Mon Feb 5 07:34:54 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46400000000000002,"15min":0.46000000000000002},"min":{"1min":0.30399999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71599999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.309,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.76300000000000001,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42699999999999999}]},{"osd":6,"last update":"Mon Feb 5 07:34:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.63900000000000001}]},{"osd":7,"last update":"Mon Feb 5 07:34:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.70499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-05T07:35:24.931 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-05T07:35:24.931 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-05T07:35:24.931 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-05T07:35:24.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921 shell --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 -- ceph health --format=json 2024-02-05T07:35:25.187 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/mon.a/config 2024-02-05T07:35:25.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[29879]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:25.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[29879]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:35:25.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:25.308 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:25.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[41444]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:25.308 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[41444]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:35:25.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:25.309 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:25 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:25.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:25 smithi050 ceph-mon[33492]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:25.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:25 smithi050 ceph-mon[33492]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-05T07:35:25.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:25 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:25.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:25 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:26.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 -- 172.21.15.43:0/1933469502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05201070a0 msgr2=0x7f052010ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1933469502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05201070a0 0x7f052010ec20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0518009f90 tx=0x7f051800b330 comp rx=0 tx=0).stop 2024-02-05T07:35:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 -- 172.21.15.43:0/1933469502 shutdown_connections 2024-02-05T07:35:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1933469502 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f05201070a0 0x7f052010ec20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1933469502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f052006a4c0 0x7f052006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1933469502 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052006ae70 0x7f0520106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 -- 172.21.15.43:0/1933469502 >> 172.21.15.43:0/1933469502 conn(0x7f0520076c70 msgr2=0x7f0520077080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 -- 172.21.15.43:0/1933469502 shutdown_connections 2024-02-05T07:35:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.806+0000 7f0524cb7700 1 -- 172.21.15.43:0/1933469502 wait complete. 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 Processor -- start 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 -- start start 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f052006a4c0 0x7f0520100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052006ae70 0x7f0520101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05201070a0 0x7f05201017a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05201a5d20 con 0x7f05201070a0 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05201a5e90 con 0x7f052006ae70 2024-02-05T07:35:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f0524cb7700 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05201a6050 con 0x7f052006a4c0 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f051e59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f052006a4c0 0x7f0520100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f051e59c700 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f052006a4c0 0x7f0520100d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:58234/0 (socket says 172.21.15.43:58234) 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f051dd9b700 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052006ae70 0x7f0520101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f051e59c700 1 -- 172.21.15.43:0/1029393082 learned_addr learned my addr 172.21.15.43:0/1029393082 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.807+0000 7f051ed9d700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05201070a0 0x7f05201017a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f051e59c700 1 -- 172.21.15.43:0/1029393082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052006ae70 msgr2=0x7f0520101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f051e59c700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052006ae70 0x7f0520101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f051e59c700 1 -- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05201070a0 msgr2=0x7f05201017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f051e59c700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05201070a0 0x7f05201017a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f051e59c700 1 -- 172.21.15.43:0/1029393082 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05100095f0 con 0x7f052006a4c0 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f051dd9b700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052006ae70 0x7f0520101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T07:35:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f051e59c700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f052006a4c0 0x7f0520100d20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f051000ec10 tx=0x7f051000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f050f7fe700 1 -- 172.21.15.43:0/1029393082 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051000cc90 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f050f7fe700 1 -- 172.21.15.43:0/1029393082 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f051000cdf0 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0518009c40 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f050f7fe700 1 -- 172.21.15.43:0/1029393082 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0510003df0 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.808+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05201a66a0 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.809+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052010a290 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.809+0000 7f050f7fe700 1 -- 172.21.15.43:0/1029393082 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f0510004000 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.810+0000 7f050f7fe700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f050806dac0 0x7f050806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.810+0000 7f051dd9b700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f050806dac0 0x7f050806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.810+0000 7f050f7fe700 1 -- 172.21.15.43:0/1029393082 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7508+0+0 (secure 0 0 0) 0x7f0510014070 con 0x7f052006a4c0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.810+0000 7f051dd9b700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f050806dac0 0x7f050806ff80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0520100900 tx=0x7f0514009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T07:35:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:26.814+0000 7f050f7fe700 1 -- 172.21.15.43:0/1029393082 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0510057110 con 0x7f052006a4c0 2024-02-05T07:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.062+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0520102250 con 0x7f052006a4c0 2024-02-05T07:35:27.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.062+0000 7f050f7fe700 1 -- 172.21.15.43:0/1029393082 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f051005a7d0 con 0x7f052006a4c0 2024-02-05T07:35:27.064 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-02-05T07:35:27.064 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f050806dac0 msgr2=0x7f050806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f050806dac0 0x7f050806ff80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0520100900 tx=0x7f0514009260 comp rx=0 tx=0).stop 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f052006a4c0 msgr2=0x7f0520100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f052006a4c0 0x7f0520100d20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f051000ec10 tx=0x7f051000ef20 comp rx=0 tx=0).stop 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 shutdown_connections 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05201070a0 0x7f05201017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f050806dac0 0x7f050806ff80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f052006a4c0 0x7f0520100d20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 --2- 172.21.15.43:0/1029393082 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f052006ae70 0x7f0520101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.064+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 >> 172.21.15.43:0/1029393082 conn(0x7f0520076c70 msgr2=0x7f05200ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.065+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 shutdown_connections 2024-02-05T07:35:27.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-02-05T07:35:27.065+0000 7f0524cb7700 1 -- 172.21.15.43:0/1029393082 wait complete. 2024-02-05T07:35:27.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:27 smithi043 ceph-mon[29879]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:27.362 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1029393082' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T07:35:27.362 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:27 smithi043 ceph-mon[41444]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:27.362 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1029393082' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T07:35:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:27 smithi050 ceph-mon[33492]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1029393082' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-05T07:35:27.769 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-05T07:35:27.770 INFO:tasks.cephadm:Setup complete, yielding 2024-02-05T07:35:27.770 INFO:teuthology.run_tasks:Running task workunit... 2024-02-05T07:35:27.789 INFO:tasks.workunit:Pulling workunits from ref 164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:35:27.789 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-05T07:35:27.789 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-05T07:35:27.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T07:35:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-05T07:35:27.807 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-05T07:35:27.862 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-05T07:35:27.862 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-05T07:35:27.918 INFO:tasks.workunit:timeout=3h 2024-02-05T07:35:27.919 INFO:tasks.workunit:cleanup=True 2024-02-05T07:35:27.919 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 164d3be4a527be775f4d3c343ea591122f1b8921 2024-02-05T07:35:27.975 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-05T07:35:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:29 smithi043 ceph-mon[29879]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:35:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:35:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:29 smithi043 ceph-mon[41444]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:29 smithi050 ceph-mon[33492]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:31.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:35:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:35:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:31 smithi043 ceph-mon[29879]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:31.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:31 smithi043 ceph-mon[41444]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:31 smithi050 ceph-mon[33492]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:32.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:32.313 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:31 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:31.946Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003279317s 2024-02-05T07:35:32.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:33.395 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:33 smithi050 ceph-mon[33492]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:33 smithi043 ceph-mon[29879]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:33 smithi043 ceph-mon[41444]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:35.282 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-mon[33492]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:35.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:35 smithi043 ceph-mon[29879]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:35.365 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:35 smithi043 ceph-mon[41444]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:35.573 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 podman[66900]: 2024-02-05 07:35:35.495939132 +0000 UTC m=+0.115797728 container create b2b6208640ac1a577734d837f99a3ff6c75a8e22a4b858963fa8fb415f49dc6c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a, version=8.5, io.buildah.version=1.32.0, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, maintainer=Paul Cuzner , release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, name=ubi8, architecture=x86_64, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, vendor=Red Hat, Inc.) 2024-02-05T07:35:36.007 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 podman[66900]: 2024-02-05 07:35:35.757223984 +0000 UTC m=+0.377082582 container init b2b6208640ac1a577734d837f99a3ff6c75a8e22a4b858963fa8fb415f49dc6c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a, vendor=Red Hat, Inc., io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, maintainer=Paul Cuzner , vcs-type=git, description=Ceph Grafana Container, architecture=x86_64, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.tags=base rhel8, io.openshift.expose-services=, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, distribution-scope=public) 2024-02-05T07:35:36.007 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 podman[66900]: 2024-02-05 07:35:35.762585115 +0000 UTC m=+0.382443722 container start b2b6208640ac1a577734d837f99a3ff6c75a8e22a4b858963fa8fb415f49dc6c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a, name=ubi8, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, com.redhat.component=ubi8-container, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, io.buildah.version=1.32.0, io.openshift.expose-services=, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., distribution-scope=public, vendor=Red Hat, Inc., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner ) 2024-02-05T07:35:36.008 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 bash[66900]: b2b6208640ac1a577734d837f99a3ff6c75a8e22a4b858963fa8fb415f49dc6c 2024-02-05T07:35:36.008 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 systemd[1]: Started Ceph grafana.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:35:36.008 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920678842Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-05T07:35:36.008 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920870137Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-05T07:35:36.008 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920889532Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-05T07:35:36.008 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920897672Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920904372Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920911403Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920917451Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920923634Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920929776Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920935715Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920941871Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920951144Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920957114Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=settings t=2024-02-05T07:35:35.920963103Z level=info msg="App mode production" 2024-02-05T07:35:36.009 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=sqlstore t=2024-02-05T07:35:35.921006153Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=sqlstore t=2024-02-05T07:35:35.921023548Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.921858381Z level=info msg="Starting DB migrations" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.92194591Z level=info msg="Executing migration" id="create migration_log table" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.922498997Z level=info msg="Executing migration" id="create user table" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.922932423Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.923373695Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.923763164Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.924339514Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.924709819Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.925366137Z level=info msg="Executing migration" id="create user table v2" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.925745356Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.926108991Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-05T07:35:36.010 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.926515524Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.926853427Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.927150663Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.927514023Z level=info msg="Executing migration" id="Update user table charset" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.927784017Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.928132035Z level=info msg="Executing migration" id="Add missing user data" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.92841785Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.928815244Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.929168773Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.929528153Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.9306288Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.931026876Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.931430671Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.931821022Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-05T07:35:36.011 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.932236152Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.93261324Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.932871687Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.93319084Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.93351267Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.933868299Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.934184105Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.934809658Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.935173047Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.935510613Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.935873839Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.936198553Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.936586848Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.936894488Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-05T07:35:36.012 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.937170372Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-05T07:35:36.013 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.937438681Z level=info msg="Executing migration" id="create star table" 2024-02-05T07:35:36.013 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.937788429Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-05T07:35:36.013 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.938170865Z level=info msg="Executing migration" id="create org table v1" 2024-02-05T07:35:36.013 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.938556794Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-05T07:35:36.013 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.938954326Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-05T07:35:36.013 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.939299687Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-05T07:35:36.014 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.939713986Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-05T07:35:36.014 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.940099047Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.940453299Z level=info msg="Executing migration" id="Update org table charset" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.940756577Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.940984587Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.941237025Z level=info msg="Executing migration" id="create dashboard table" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.941578525Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.941973366Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.942357332Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.942720435Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.943117498Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.943440513Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.944282839Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.944658863Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-05T07:35:36.015 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.945055099Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.945456188Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.945764005Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.946086486Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.9463174Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.946797379Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.947284469Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.947740719Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.948132895Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.948554305Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.948956074Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.949293556Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.949522951Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-05T07:35:36.016 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.949779517Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.950266182Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.950744905Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.951213731Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.951691981Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.951991993Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.952389971Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.952755614Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.953013008Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.953360444Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.953717802Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.954618849Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-05T07:35:36.017 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.954999921Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.955393088Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.955779644Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.956093937Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.95638005Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.956867133Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.957256435Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.957508987Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.957785816Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.958166671Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.958679271Z level=info msg="Executing migration" id="create data_source table" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.959070573Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.959499589Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-05T07:35:36.018 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.959901083Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.960249393Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.960649125Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.961693817Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.962091677Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.9624825Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.962899841Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.963225035Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.963766881Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.964282507Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.964582066Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.964886933Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-05T07:35:36.019 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.965375509Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.965706415Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.966012853Z level=info msg="Executing migration" id="Add uid column" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.96652835Z level=info msg="Executing migration" id="Update uid value" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.966847695Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.967246536Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.967678873Z level=info msg="Executing migration" id="create api_key table" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.968116895Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.968536786Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.968938368Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.969377091Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.969769747Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-05T07:35:36.020 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.970153853Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.97051702Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.971559462Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.971957162Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.972323677Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.972759493Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.973176662Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.973464738Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.973866115Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.974105037Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.974637376Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.975153124Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.975423568Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.975950944Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.976450941Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-05T07:35:36.021 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.976824254Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.977142437Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.977494365Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.977935396Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.978423606Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.978856859Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.979108756Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.979344153Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.979899021Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.980426547Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.9807298Z level=info msg="Executing migration" id="create quota table v1" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.981098499Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.98151543Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-05T07:35:36.022 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.981794535Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.982254252Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.982666486Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.983210684Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.983458187Z level=info msg="Executing migration" id="create session table" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.983937491Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.98419521Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.984444599Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.98483931Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.985197999Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.985434469Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.985719667Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.985970642Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.986218971Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.986615688Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-05T07:35:36.023 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.986888115Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.987427017Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.987733391Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.988278366Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.988856804Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.989138509Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.989559327Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.990067983Z level=info msg="Executing migration" id="create alert table v1" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.990496654Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.990939083Z level=info msg="Executing migration" id="add index alert state" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.991326268Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.991725744Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.992070258Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-05T07:35:36.024 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.992463776Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.992851647Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.994262234Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.994628819Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.995027753Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.995290592Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.995627624Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.99598492Z level=info msg="Executing migration" id="Add column is_default" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.996547249Z level=info msg="Executing migration" id="Add column frequency" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.997147747Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.997769307Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-05T07:35:36.025 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.998371484Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.998775932Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.999031356Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.999273102Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:35 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:35.999648335Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.000049115Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.000370349Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.000754648Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.00115256Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.001785272Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.00239051Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.002681969Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.003086101Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.003423136Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-05T07:35:36.026 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.004038617Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.004277656Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.004682504Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.005093432Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.00533946Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.005793353Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.006181607Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.006576658Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.006964997Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.007377887Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.007834009Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.00811583Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.008846431Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.009266092Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.009948995Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-05T07:35:36.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.010336808Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-05T07:35:36.028 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.010784537Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-05T07:35:36.028 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.011186589Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-05T07:35:36.028 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.012810948Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-05T07:35:36.262 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.013282705Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-05T07:35:36.262 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.013762632Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-05T07:35:36.262 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.014118053Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-05T07:35:36.262 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.014433811Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-05T07:35:36.262 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.014788131Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.015420156Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.016087965Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.016563186Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.017148849Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.017516311Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.01836772Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.018897752Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.019252891Z level=info msg="Executing migration" id="Move region to single row" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.019725198Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-05T07:35:36.263 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.020124778Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.020492158Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.020893271Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.021266503Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.021641542Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.022082355Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.02237431Z level=info msg="Executing migration" id="create test_data table" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.022792808Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-05T07:35:36.264 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.023156789Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.02355835Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.023967338Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.024219911Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.024486038Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.024751616Z level=info msg="Executing migration" id="create team table" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.025189047Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.025630288Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.026124564Z level=info msg="Executing migration" id="create team member table" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.026578895Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.027054433Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-05T07:35:36.265 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.027558278Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.028026897Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.02885465Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.029726481Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.030594627Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.031056259Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.031594207Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.032069314Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.032467939Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.03294085Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.033332943Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.03373402Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.034126671Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.034470275Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-05T07:35:36.266 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.03479407Z level=info msg="Executing migration" id="create tag table" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.035143569Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.035538499Z level=info msg="Executing migration" id="create login attempt table" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.035903552Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.036289121Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.036686961Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.038541067Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.038908964Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-05T07:35:36.267 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.039307119Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.03961704Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.039952409Z level=info msg="Executing migration" id="create user auth table" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.040275057Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.040687278Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.040984446Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.041700271Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.042410421Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.043174605Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.043918664Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.044336316Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-05T07:35:36.269 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.045073117Z level=info msg="Executing migration" id="create server_lock table" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.045442672Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.045880466Z level=info msg="Executing migration" id="create user auth token table" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.046293171Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.046724235Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.047141987Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.047572877Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.048319838Z level=info msg="Executing migration" id="create cache_data table" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.048720388Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.04916883Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.049573094Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.049998192Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.050246532Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.050503686Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.05113998Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-05T07:35:36.270 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.051584508Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.052006034Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.052277402Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.052699683Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.053106037Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.053512815Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.053934063Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.054734167Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.055145385Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.055405739Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.055822556Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.05622188Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.056670326Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.056953163Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-05T07:35:36.271 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.057330156Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.057735121Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.058154472Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.058586563Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.059355365Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.059793362Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.060235693Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.063685188Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.067214596Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.067685863Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.06810823Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.068925944Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.069322136Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.069787634Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-05T07:35:36.272 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.070219501Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.07064872Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.070934077Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.071744792Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.072535497Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.073324428Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.07375091Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.074179577Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.07500298Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.075812294Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.076240317Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.077050894Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.07786888Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.078157414Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-05T07:35:36.273 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.07858232Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.079014942Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.079437769Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.079734184Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.080637418Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.081445351Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.082282575Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.083104021Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.0839288Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.084210982Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.084595425Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.085406916Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.085715029Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.086575131Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-05T07:35:36.274 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.087003095Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.087832566Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.088203466Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.088638211Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.089491284Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.089876596Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.090309345Z level=info msg="Executing migration" id="create alert_image table" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.090691311Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.091126647Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.091375694Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.0917832Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.092212319Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.092301646Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.092533585Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-05T07:35:36.275 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.092883045Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.093260735Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.093387744Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.093528357Z level=info msg="alerts found to migrate" alerts=0 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.093916622Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.094370933Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.09485599Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.09524479Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.09567901Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.096124057Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.096367013Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.096732542Z level=info msg="Executing migration" id="create data_keys table" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.097138171Z level=info msg="Executing migration" id="create secrets table" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.097567446Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-05T07:35:36.276 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.10192845Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.102832645Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.10312546Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.107414077Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.111731251Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.112120014Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.112537933Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.112886624Z level=info msg="Executing migration" id="create permission table" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.113257298Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.113706203Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.114131301Z level=info msg="Executing migration" id="create role table" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.114516167Z level=info msg="Executing migration" id="add column display_name" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.115408433Z level=info msg="Executing migration" id="add column group_name" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.116317054Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-05T07:35:36.277 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.116749954Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.117167692Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.117601033Z level=info msg="Executing migration" id="create team role table" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.118001602Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.118412661Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.118888222Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.119322637Z level=info msg="Executing migration" id="create user role table" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.119715677Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.120176067Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.120602569Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.121018148Z level=info msg="Executing migration" id="create builtin role table" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.1214171Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.12185083Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.122258954Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-05T07:35:36.278 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.123227483Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.12370106Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.124120891Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.124510511Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.124926053Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.125293112Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.125728403Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.126697089Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.127085313Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.127510037Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.127804905Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.128154813Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.128593654Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.128972858Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-05T07:35:36.279 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.129231796Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.12955024Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.12992708Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.130343123Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.131337158Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.131650687Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.13204911Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.132459272Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.132904923Z level=info msg="Executing migration" id="add correlation config column" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.133887375Z level=info msg="Executing migration" id="create entity_events table" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.134307828Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.134742018Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.134839111Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-05T07:35:36.280 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.135109605Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.135200894Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.135422412Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.135828258Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.13618917Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.136619091Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.137045452Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.137416388Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.137850442Z level=info msg="Executing migration" id="Drop public config table" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.138226992Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.138645062Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.139090209Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.13951039Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.139936484Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.143025987Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-05T07:35:36.281 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.144026661Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.145016951Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.145319061Z level=info msg="Executing migration" id="add share column" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.146299638Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.146751675Z level=info msg="Executing migration" id="create file table" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.147169699Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.147592847Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.148045805Z level=info msg="Executing migration" id="create file_meta table" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.148407725Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.14883378Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.149109479Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.14943281Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.149744091Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-05T07:35:36.282 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.150115295Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.151121431Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.151411917Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.151862633Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.152196814Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.152461497Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.15281947Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.153829827Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.154828584Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.155231799Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.165268263Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.165888299Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.166313192Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.169871836Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-05T07:35:36.283 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.170179089Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.170515296Z level=info msg="Executing migration" id="create folder table" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.170898582Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.171310281Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.171732938Z level=info msg="Executing migration" id="Update folder title length" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.172003707Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=migrator t=2024-02-05T07:35:36.172402771Z level=info msg="migrations completed" performed=485 skipped=0 duration=250.487692ms 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=sqlstore t=2024-02-05T07:35:36.172816751Z level=info msg="Created default organization" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=plugin.loader t=2024-02-05T07:35:36.232086355Z level=info msg="Plugin registered" pluginID=input 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=plugin.loader t=2024-02-05T07:35:36.261286104Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=plugin.loader t=2024-02-05T07:35:36.261306066Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=secrets t=2024-02-05T07:35:36.261339585Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=query_data t=2024-02-05T07:35:36.264608548Z level=info msg="Query Service initialization" 2024-02-05T07:35:36.284 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=live.push_http t=2024-02-05T07:35:36.268387677Z level=info msg="Live Push Gateway initialization" 2024-02-05T07:35:37.190 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:36 smithi043 systemd[1]: Starting Ceph node-exporter.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:35:37.190 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[29879]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[29879]: Deploying daemon node-exporter.a on smithi043 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[41444]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.191 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:36 smithi043 ceph-mon[41444]: Deploying daemon node-exporter.a on smithi043 2024-02-05T07:35:37.251 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-mon[33492]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:37.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:37.252 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-mon[33492]: Deploying daemon node-exporter.a on smithi043 2024-02-05T07:35:37.252 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=infra.usagestats.collector t=2024-02-05T07:35:36.915125519Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-05T07:35:37.252 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=provisioning.datasources t=2024-02-05T07:35:36.916077067Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-05T07:35:37.252 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=provisioning.datasources t=2024-02-05T07:35:36.927205217Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=provisioning.alerting t=2024-02-05T07:35:36.938368789Z level=info msg="starting to provision alerting" 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=provisioning.alerting t=2024-02-05T07:35:36.938387644Z level=info msg="finished to provision alerting" 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=ngalert.state.manager t=2024-02-05T07:35:36.938551415Z level=info msg="Warming state cache for startup" 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=grafanaStorageLogger t=2024-02-05T07:35:36.938557769Z level=info msg="storage starting" 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=http.server t=2024-02-05T07:35:36.941055909Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=ngalert.state.manager t=2024-02-05T07:35:36.94445933Z level=info msg="State cache has been initialized" states=0 duration=5.907411ms 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=ngalert.multiorg.alertmanager t=2024-02-05T07:35:36.944499599Z level=info msg="starting MultiOrg Alertmanager" 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=ticker t=2024-02-05T07:35:36.944526155Z level=info msg=starting first_tick=2024-02-05T07:35:40Z 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=sqlstore.transactions t=2024-02-05T07:35:36.94903827Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=sqlstore.transactions t=2024-02-05T07:35:36.959420759Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-02-05T07:35:37.253 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:35:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=sqlstore.transactions t=2024-02-05T07:35:36.969738508Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-02-05T07:35:37.563 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:37 smithi043 bash[79797]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-05T07:35:38.063 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:37 smithi043 bash[79797]: Getting image source signatures 2024-02-05T07:35:38.063 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:37 smithi043 bash[79797]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-05T07:35:38.063 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:37 smithi043 bash[79797]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-05T07:35:38.063 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:37 smithi043 bash[79797]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-05T07:35:39.164 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:38 smithi043 ceph-mon[41444]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:39.164 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:38 smithi043 bash[79797]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-05T07:35:39.164 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:38 smithi043 ceph-mon[29879]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:39.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:38 smithi050 ceph-mon[33492]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:39.459 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:35:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:35:39.459 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 bash[79797]: Writing manifest to image destination 2024-02-05T07:35:39.459 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 podman[79797]: 2024-02-05 07:35:39.377124165 +0000 UTC m=+2.307015201 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 podman[79797]: 2024-02-05 07:35:39.458779126 +0000 UTC m=+2.388670196 container create 6d8340307e3272af692d04d68d973fe624ff5f60224c5d98dc1f18b73b4c5d59 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 podman[79797]: 2024-02-05 07:35:39.677535521 +0000 UTC m=+2.607426552 container init 6d8340307e3272af692d04d68d973fe624ff5f60224c5d98dc1f18b73b4c5d59 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 podman[79797]: 2024-02-05 07:35:39.682985483 +0000 UTC m=+2.612876523 container start 6d8340307e3272af692d04d68d973fe624ff5f60224c5d98dc1f18b73b4c5d59 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.687Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.687Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.687Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.687Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-05T07:35:39.778 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-05T07:35:39.779 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-05T07:35:39.779 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-05T07:35:39.779 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=arp 2024-02-05T07:35:39.779 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-05T07:35:39.779 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-05T07:35:39.779 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-05T07:35:39.779 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=edac 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-05T07:35:39.780 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=os 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-05T07:35:39.781 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=stat 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=time 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=uname 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-05T07:35:39.782 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-05T07:35:39.783 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a[79849]: ts=2024-02-05T07:35:39.688Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-05T07:35:39.783 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 bash[79797]: 6d8340307e3272af692d04d68d973fe624ff5f60224c5d98dc1f18b73b4c5d59 2024-02-05T07:35:40.063 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 07:35:39 smithi043 systemd[1]: Started Ceph node-exporter.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:35:40.718 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:35:41.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-mon[33492]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:41.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:41.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.033 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:40 smithi050 ceph-mon[33492]: Deploying daemon node-exporter.b on smithi050 2024-02-05T07:35:41.033 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:40 smithi050 systemd[1]: Starting Ceph node-exporter.b for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[41444]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[41444]: Deploying daemon node-exporter.b on smithi050 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[29879]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:40 smithi043 ceph-mon[29879]: Deploying daemon node-exporter.b on smithi050 2024-02-05T07:35:41.322 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:41 smithi050 bash[67184]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-05T07:35:41.723 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:41 smithi050 bash[67184]: Getting image source signatures 2024-02-05T07:35:41.723 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:41 smithi050 bash[67184]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-05T07:35:41.723 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:41 smithi050 bash[67184]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-05T07:35:41.723 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:41 smithi050 bash[67184]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-05T07:35:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:41 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:42.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:41 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:41 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:42 smithi043 ceph-mon[29879]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:42 smithi043 ceph-mon[41444]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:42 smithi050 ceph-mon[33492]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:43.072 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:42 smithi050 bash[67184]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-05T07:35:43.072 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:42 smithi050 bash[67184]: Writing manifest to image destination 2024-02-05T07:35:43.572 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 podman[67184]: 2024-02-05 07:35:43.224610492 +0000 UTC m=+2.288033987 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-05T07:35:43.572 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 podman[67184]: 2024-02-05 07:35:43.331546906 +0000 UTC m=+2.394970393 container create 536d1af5c6fd912559534524f7f4ef28ead1b5a4d7a9248475695c1f2db21978 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 podman[67184]: 2024-02-05 07:35:43.592070487 +0000 UTC m=+2.655493974 container init 536d1af5c6fd912559534524f7f4ef28ead1b5a4d7a9248475695c1f2db21978 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 podman[67184]: 2024-02-05 07:35:43.597824821 +0000 UTC m=+2.661248306 container start 536d1af5c6fd912559534524f7f4ef28ead1b5a4d7a9248475695c1f2db21978 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.601Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.601Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.602Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.602Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.602Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-05T07:35:44.073 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=arp 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-05T07:35:44.074 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=edac 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-05T07:35:44.075 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=os 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=stat 2024-02-05T07:35:44.076 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=time 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=uname 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b[67235]: ts=2024-02-05T07:35:43.603Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 bash[67184]: 536d1af5c6fd912559534524f7f4ef28ead1b5a4d7a9248475695c1f2db21978 2024-02-05T07:35:44.077 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 07:35:43 smithi050 systemd[1]: Started Ceph node-exporter.b for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:35:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:35:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:35:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:46 smithi043 ceph-mon[29879]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:46 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:46 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:46 smithi043 ceph-mon[41444]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:46 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:46 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:46 smithi050 ceph-mon[33492]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:46 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:46 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:49.210 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:48 smithi043 ceph-mon[29879]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:49.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:48 smithi043 ceph-mon[41444]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:49.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:48 smithi050 ceph-mon[33492]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:35:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:35:50.959 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:35:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:35:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:50 smithi043 ceph-mon[29879]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:50 smithi043 ceph-mon[41444]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:51.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:50 smithi050 ceph-mon[33492]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:52.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:51 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:51 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:51 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:35:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:52 smithi043 ceph-mon[29879]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:52 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:52 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:52 smithi043 ceph-mon[41444]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:52 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:52 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:52 smithi050 ceph-mon[33492]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:52 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:52 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:54.273 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:53 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:35:54.274 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:53 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:35:54.274 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:53 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:54.274 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:53 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:35:54.274 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:53 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:35:54.274 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:53 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:53 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:35:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:53 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:35:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:53 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:55.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:54 smithi050 ceph-mon[33492]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-05T07:35:55.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:54 smithi050 ceph-mon[33492]: Reconfiguring daemon alertmanager.a on smithi043 2024-02-05T07:35:55.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:54 smithi050 ceph-mon[33492]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:55.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:54 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[41444]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-05T07:35:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[41444]: Reconfiguring daemon alertmanager.a on smithi043 2024-02-05T07:35:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[41444]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[29879]: Reconfiguring alertmanager.a (dependencies changed)... 2024-02-05T07:35:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[29879]: Reconfiguring daemon alertmanager.a on smithi043 2024-02-05T07:35:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[29879]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:35:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:54 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:35:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:56 smithi043 ceph-mon[29879]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:57.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:56 smithi043 ceph-mon[41444]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:57.064 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:56 smithi043 systemd[1]: Stopping Ceph alertmanager.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:35:57.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:56 smithi050 ceph-mon[33492]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:35:57.385 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:57 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[79049]: ts=2024-02-05T07:35:57.103Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-05T07:35:57.385 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:57 smithi043 podman[80670]: 2024-02-05 07:35:57.124332021 +0000 UTC m=+0.113882070 container died e67eb5621b2dec09a069419af56f4908dd82e288241d4a7641b840cd9d198f9b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:57.761 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:57 smithi043 podman[80670]: 2024-02-05 07:35:57.484339989 +0000 UTC m=+0.473890039 container cleanup e67eb5621b2dec09a069419af56f4908dd82e288241d4a7641b840cd9d198f9b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:57.761 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:57 smithi043 bash[80670]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a 2024-02-05T07:35:58.044 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:57 smithi043 podman[80688]: 2024-02-05 07:35:57.760921057 +0000 UTC m=+0.635350300 container remove e67eb5621b2dec09a069419af56f4908dd82e288241d4a7641b840cd9d198f9b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:58.045 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:57 smithi043 podman[80688]: 2024-02-05 07:35:57.844809797 +0000 UTC m=+0.719239037 volume remove a5655f1ed081c2f6656d769608861139bb36f4bbdeb4ad4623a81726a0bd2d77 2024-02-05T07:35:58.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-05T07:35:58.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 systemd[1]: Stopped Ceph alertmanager.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:35:58.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 systemd[1]: Starting Ceph alertmanager.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:35:58.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 podman[80786]: 2024-02-05 07:35:58.175801816 +0000 UTC m=+0.017628239 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-05T07:35:58.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 podman[80786]: 2024-02-05 07:35:58.304050787 +0000 UTC m=+0.145877212 volume create e20146e562f42b8bad4cad482897bab7ea0dc22d61e067b0d0d8f1e4f16c1490 2024-02-05T07:35:58.396 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 podman[80786]: 2024-02-05 07:35:58.396241964 +0000 UTC m=+0.238068389 container create 2c6ac989f290818a6b10833107a90a2b7c46511842bcc6514571a0b3b3c90bb3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:58.971 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 podman[80786]: 2024-02-05 07:35:58.690406501 +0000 UTC m=+0.532232931 container init 2c6ac989f290818a6b10833107a90a2b7c46511842bcc6514571a0b3b3c90bb3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:58.971 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 podman[80786]: 2024-02-05 07:35:58.696118094 +0000 UTC m=+0.537944519 container start 2c6ac989f290818a6b10833107a90a2b7c46511842bcc6514571a0b3b3c90bb3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.709Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.709Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.710Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.711Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.734Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.735Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.737Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:35:58.737Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 bash[80786]: 2c6ac989f290818a6b10833107a90a2b7c46511842bcc6514571a0b3b3c90bb3 2024-02-05T07:35:58.972 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:35:58 smithi043 systemd[1]: Started Ceph alertmanager.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:35:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-mon[29879]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:35:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:59.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:35:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:35:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:35:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-mon[41444]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:35:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:58 smithi050 ceph-mon[33492]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:35:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:35:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:59 smithi043 ceph-mon[29879]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-05T07:36:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:35:59 smithi043 ceph-mon[29879]: Reconfiguring daemon prometheus.a on smithi050 2024-02-05T07:36:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:59 smithi043 ceph-mon[41444]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-05T07:36:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:35:59 smithi043 ceph-mon[41444]: Reconfiguring daemon prometheus.a on smithi050 2024-02-05T07:36:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:59 smithi050 ceph-mon[33492]: Reconfiguring prometheus.a (dependencies changed)... 2024-02-05T07:36:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:35:59 smithi050 ceph-mon[33492]: Reconfiguring daemon prometheus.a on smithi050 2024-02-05T07:36:00.976 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:36:00 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:36:00.711Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000575146s 2024-02-05T07:36:00.977 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:36:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:36:01.295 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:00 smithi050 ceph-mon[33492]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:36:01.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:00 smithi043 ceph-mon[29879]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:36:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:00 smithi043 ceph-mon[41444]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:36:02.202 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:01 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:02.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:01 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:01 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:02.823 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 systemd[1]: Stopping Ceph prometheus.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:36:03.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-mon[33492]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.899Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-05T07:36:03.207 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[64907]: ts=2024-02-05T07:36:02.900Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-05T07:36:03.208 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:02 smithi050 podman[68287]: 2024-02-05 07:36:02.91033893 +0000 UTC m=+0.120640636 container died b2ee6c55bd7fd69b9d600bc7a65e5287a388670a58f340b793232964fb7680ff (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:36:03.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:02 smithi043 ceph-mon[29879]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:03.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:02 smithi043 ceph-mon[41444]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:03.556 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 podman[68287]: 2024-02-05 07:36:03.313582001 +0000 UTC m=+0.523883706 container cleanup b2ee6c55bd7fd69b9d600bc7a65e5287a388670a58f340b793232964fb7680ff (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:36:03.556 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 bash[68287]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a 2024-02-05T07:36:03.822 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 podman[68306]: 2024-02-05 07:36:03.556455121 +0000 UTC m=+0.646178247 container remove b2ee6c55bd7fd69b9d600bc7a65e5287a388670a58f340b793232964fb7680ff (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:36:03.822 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-02-05T07:36:03.823 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 systemd[1]: Stopped Ceph prometheus.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:36:03.823 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@prometheus.a.service: Consumed 1.151s CPU time. 2024-02-05T07:36:03.823 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 systemd[1]: Starting Ceph prometheus.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T07:36:04.160 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 podman[68409]: 2024-02-05 07:36:03.793216073 +0000 UTC m=+0.018483552 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-05T07:36:04.160 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:03 smithi050 podman[68409]: 2024-02-05 07:36:03.907966977 +0000 UTC m=+0.133234492 container create 5afd0d3c75b9f14961c9539226ac02f7a079bddabcbba404402d2bc09b291822 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:36:04.160 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 podman[68409]: 2024-02-05 07:36:04.160317028 +0000 UTC m=+0.385584508 container init 5afd0d3c75b9f14961c9539226ac02f7a079bddabcbba404402d2bc09b291822 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:36:04.573 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 podman[68409]: 2024-02-05 07:36:04.165487697 +0000 UTC m=+0.390755172 container start 5afd0d3c75b9f14961c9539226ac02f7a079bddabcbba404402d2bc09b291822 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T07:36:04.573 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.190Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-05T07:36:04.573 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.190Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-05T07:36:04.573 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.190Z caller=main.go:570 level=info host_details="(Linux 5.14.0-416.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024 x86_64 smithi050 (none))" 2024-02-05T07:36:04.573 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.190Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-05T07:36:04.573 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.190Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-05T07:36:04.573 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.192Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.192Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.193Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.193Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.194Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.194Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.553µs 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.194Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.196Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.196Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.196Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=22.409µs wal_replay_duration=1.789497ms wbl_replay_duration=114ns total_replay_duration=1.833228ms 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.197Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.197Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-05T07:36:04.574 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.197Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-05T07:36:04.575 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.217Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.87882ms db_storage=711ns remote_storage=1.107µs web_handler=299ns query_engine=856ns scrape=461.528µs scrape_sd=75.736µs notify=34.082µs notify_sd=31.008µs rules=19.061632ms tracing=6.151µs 2024-02-05T07:36:04.575 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.217Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-05T07:36:04.575 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T07:36:04.217Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-05T07:36:04.575 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 bash[68409]: 5afd0d3c75b9f14961c9539226ac02f7a079bddabcbba404402d2bc09b291822 2024-02-05T07:36:04.575 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 07:36:04 smithi050 systemd[1]: Started Ceph prometheus.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STOPPING 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STOPPED 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STARTING 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Serving on http://:::9283 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STARTED 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STOPPING 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STOPPED 2024-02-05T07:36:04.697 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STARTING 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Serving on http://:::9283 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STARTED 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STOPPING 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STOPPED 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STARTING 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Serving on http://:::9283 2024-02-05T07:36:04.698 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:07:36:04] ENGINE Bus STARTED 2024-02-05T07:36:05.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:36:05.310 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi050.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi050.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:36:05.311 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi050.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi050.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:36:05.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:36:05.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi050.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi050.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:05.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:36:06.297 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:05 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:36:06.297 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:05 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:36:06.297 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:05 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:36:06.297 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:05 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi050.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:36:06.298 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:05 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:36:06.298 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:05 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi050.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:36:06.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:36:06.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:36:06.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:36:06.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi050.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:36:06.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:36:06.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi050.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:36:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-05T07:36:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-02-05T07:36:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-05T07:36:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi050.front.sepia.ceph.com:9095"}]: dispatch 2024-02-05T07:36:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-05T07:36:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:05 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi050.front.sepia.ceph.com:3000"}]: dispatch 2024-02-05T07:36:06.990 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:06 smithi043 ceph-mon[29879]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:06.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:06 smithi043 ceph-mon[41444]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:07.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:06 smithi050 ceph-mon[33492]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:08 smithi043 ceph-mon[29879]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:08 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:08 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:09.064 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 07:36:08 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T07:36:08.714Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003686447s 2024-02-05T07:36:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:08 smithi043 ceph-mon[41444]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:08 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:08 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:09.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:08 smithi050 ceph-mon[33492]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:09.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:08 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:09.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:08 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:36:10.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:36:10.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:09 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:10.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:09 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:09 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:09 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:36:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:09 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:36:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:09 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:09 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:10.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:36:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:36:11.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:10 smithi043 ceph-mon[29879]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:11.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:10 smithi043 ceph-mon[41444]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:11.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:10 smithi050 ceph-mon[33492]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:11 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:12.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:11 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:12.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:11 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:12 smithi043 ceph-mon[29879]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:13.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:12 smithi043 ceph-mon[41444]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:12 smithi050 ceph-mon[33492]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:14 smithi043 ceph-mon[29879]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:14 smithi043 ceph-mon[41444]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:15.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:14 smithi050 ceph-mon[33492]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:16 smithi043 ceph-mon[29879]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:17.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:16 smithi043 ceph-mon[41444]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:16 smithi050 ceph-mon[33492]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:19.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:36:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:36:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:18 smithi043 ceph-mon[29879]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:18 smithi043 ceph-mon[41444]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:18 smithi050 ceph-mon[33492]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:20.972 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:36:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:36:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:20 smithi043 ceph-mon[29879]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:21.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:20 smithi043 ceph-mon[41444]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:21.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:20 smithi050 ceph-mon[33492]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:22 smithi043 ceph-mon[41444]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:22 smithi043 ceph-mon[29879]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:22 smithi050 ceph-mon[33492]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:24 smithi043 ceph-mon[29879]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:24 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:24 smithi043 ceph-mon[41444]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:24 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:24 smithi050 ceph-mon[33492]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:24 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:26 smithi043 ceph-mon[41444]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:26 smithi043 ceph-mon[29879]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:26 smithi050 ceph-mon[33492]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:29.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:36:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:36:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:28 smithi043 ceph-mon[29879]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:29.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:28 smithi043 ceph-mon[41444]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:28 smithi050 ceph-mon[33492]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:30.988 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:36:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:36:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:30 smithi043 ceph-mon[29879]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:31.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:30 smithi043 ceph-mon[41444]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:31.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:30 smithi050 ceph-mon[33492]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:31 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:32.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:31 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:32.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:31 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:32 smithi043 ceph-mon[29879]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:32 smithi043 ceph-mon[41444]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:32 smithi050 ceph-mon[33492]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:35.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:35 smithi043 ceph-mon[29879]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:35.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:35 smithi043 ceph-mon[41444]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:35.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:35 smithi050 ceph-mon[33492]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:37.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:37 smithi043 ceph-mon[29879]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:37.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:37 smithi043 ceph-mon[41444]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:37.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:37 smithi050 ceph-mon[33492]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:39.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:36:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:36:39.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:39 smithi043 ceph-mon[29879]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:39.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:39 smithi043 ceph-mon[41444]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:39.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:39 smithi050 ceph-mon[33492]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:40.312 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:41.012 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:36:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:36:41.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:41 smithi043 ceph-mon[29879]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:41.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:41 smithi043 ceph-mon[41444]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:41.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:41 smithi050 ceph-mon[33492]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:41.858 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 97% (11941/12223) Updating files: 98% (11979/12223) Updating files: 99% (12101/12223) Updating files: 100% (12223/12223) Updating files: 100% (12223/12223), done. 2024-02-05T07:36:42.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:42.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:42.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to '164d3be4a527be775f4d3c343ea591122f1b8921'. 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-05T07:36:42.476 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-02-05T07:36:42.477 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at 164d3be4a52 Merge branch 'subclass-thread' of https://github.com/guits/ceph into wip-guits-testing-3-2024-02-04-2254 2024-02-05T07:36:42.478 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-05T07:36:42.517 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-05T07:36:42.519 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-05T07:36:42.519 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-05T07:36:43.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:43 smithi043 ceph-mon[29879]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:43.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:43 smithi043 ceph-mon[41444]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:43.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:43 smithi050 ceph-mon[33492]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:43.344 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-05T07:36:43.412 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-05T07:36:43.452 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-05T07:36:43.453 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-05T07:36:43.453 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-05T07:36:43.492 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-05T07:36:43.495 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-02-05T07:36:43.495 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-05T07:36:43.550 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-05T07:36:43.552 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-05T07:36:43.552 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=164d3be4a527be775f4d3c343ea591122f1b8921 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-05T07:36:43.621 INFO:tasks.workunit.client.0.smithi043.stderr:+ parallel=1 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' '' = --serial ']' 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ crimson=0 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' '' = --crimson ']' 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ color= 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -t 1 ']' 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ GTEST_OUTPUT=xml:/home/ubuntu/cephtest/archive/gtest_xml_report 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ declare -A pids 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.698 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio 2024-02-05T07:36:43.699 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio' 2024-02-05T07:36:43.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio 2024-02-05T07:36:43.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio 2024-02-05T07:36:43.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81627 2024-02-05T07:36:43.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio on pid 81627' 2024-02-05T07:36:43.700 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio on pid 81627 2024-02-05T07:36:43.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81627 2024-02-05T07:36:43.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-05T07:36:43.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.703 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio_pp 2024-02-05T07:36:43.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio_pp' 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio_pp 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.705 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio_pp 2024-02-05T07:36:43.706 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81634 2024-02-05T07:36:43.706 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio_pp on pid 81634 2024-02-05T07:36:43.709 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio_pp on pid 81634' 2024-02-05T07:36:43.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81634 2024-02-05T07:36:43.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-05T07:36:43.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io 2024-02-05T07:36:43.711 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io' 2024-02-05T07:36:43.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.715 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io 2024-02-05T07:36:43.715 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81643 2024-02-05T07:36:43.715 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io on pid 81643 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io on pid 81643' 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81643 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io_pp 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io_pp' 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io_pp 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.721 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.722 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.723 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io_pp 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81656 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io_pp on pid 81656' 2024-02-05T07:36:43.724 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io_pp on pid 81656 2024-02-05T07:36:43.729 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81656 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_asio 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_asio' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_asio 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.733 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.734 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_asio 2024-02-05T07:36:43.735 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81670 2024-02-05T07:36:43.736 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_asio on pid 81670' 2024-02-05T07:36:43.736 INFO:tasks.workunit.client.0.smithi043.stdout:test api_asio on pid 81670 2024-02-05T07:36:43.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81670 2024-02-05T07:36:43.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-05T07:36:43.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_list 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_list' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_list 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81627/exe 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.746 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_list 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.747 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81686 2024-02-05T07:36:43.748 INFO:tasks.workunit.client.0.smithi043.stdout:test api_list on pid 81686 2024-02-05T07:36:43.751 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.751 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_list on pid 81686' 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81686 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.752 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio.xml 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.753 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.754 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio: /' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81634/exe 2024-02-05T07:36:43.759 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock 2024-02-05T07:36:43.760 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock on pid 81707 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81707 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock on pid 81707' 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81707 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock_pp 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock_pp' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock_pp 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.766 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.767 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.768 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.769 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.770 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-05T07:36:43.771 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock_pp on pid 81727 2024-02-05T07:36:43.774 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock_pp 2024-02-05T07:36:43.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81643/exe 2024-02-05T07:36:43.774 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.774 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81727 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock_pp on pid 81727' 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81727 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc' 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_aio_pp.xml 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.776 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81656/exe 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io.log 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.780 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.781 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io.xml 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_io_pp.xml 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.784 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-05T07:36:43.785 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc on pid 81753 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81753 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc on pid 81753' 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81753 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io: /' 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc_pp 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc_pp' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81670/exe 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc_pp 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.794 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.796 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_asio: /' 2024-02-05T07:36:43.796 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc_pp on pid 81774 2024-02-05T07:36:43.800 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc_pp 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81774 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc_pp on pid 81774' 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81774 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_asio.xml 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.801 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_tier_pp 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_tier_pp' 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81686/exe 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.807 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.808 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81707/exe 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_list.xml 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock: /' 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock.xml 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_list: /' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.814 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.815 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_list.log 2024-02-05T07:36:43.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_tier_pp 2024-02-05T07:36:43.816 INFO:tasks.workunit.client.0.smithi043.stdout:test api_tier_pp on pid 81815 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_tier_pp 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81815 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_tier_pp on pid 81815' 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81815 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_pool 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.817 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_pool' 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.818 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.819 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.820 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_pool 2024-02-05T07:36:43.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-05T07:36:43.821 INFO:tasks.workunit.client.0.smithi043.stdout:test api_pool on pid 81830 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_pool 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81830 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_pool on pid 81830' 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81830 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots' 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81727/exe 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.825 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots 2024-02-05T07:36:43.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.825 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots on pid 81848 2024-02-05T07:36:43.826 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81848 2024-02-05T07:36:43.826 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots on pid 81848' 2024-02-05T07:36:43.826 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81848 2024-02-05T07:36:43.826 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots_pp 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots_pp' 2024-02-05T07:36:43.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots_pp 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_lock_pp.xml 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.830 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.830 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots_pp on pid 81865 2024-02-05T07:36:43.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots_pp 2024-02-05T07:36:43.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81865 2024-02-05T07:36:43.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots_pp on pid 81865' 2024-02-05T07:36:43.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81865 2024-02-05T07:36:43.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.833 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81753/exe 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.834 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.835 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.836 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat 2024-02-05T07:36:43.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81774/exe 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat 2024-02-05T07:36:43.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc_pp.xml 2024-02-05T07:36:43.839 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat on pid 81914 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81914 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat on pid 81914' 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81914 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-05T07:36:43.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.848 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.849 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc: /' 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_misc.xml 2024-02-05T07:36:43.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat_pp 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.854 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.855 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat_pp' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.856 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81815/exe 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81848/exe 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat_pp 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat_pp 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81990 2024-02-05T07:36:43.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat_pp on pid 81990' 2024-02-05T07:36:43.870 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat_pp on pid 81990 2024-02-05T07:36:43.871 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81990 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.872 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify' 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-05T07:36:43.873 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.874 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.875 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_tier_pp.xml 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.876 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots.xml 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.877 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.878 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify on pid 82016 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82016 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify on pid 82016' 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82016 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.880 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify_pp 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify_pp' 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify_pp 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.886 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify_pp on pid 82028 2024-02-05T07:36:43.889 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify_pp 2024-02-05T07:36:43.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.889 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82028 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify_pp on pid 82028' 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82028 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd' 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.893 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81914/exe 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.894 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat.xml 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81830/exe 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.895 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat: /' 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_pool.xml 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_pool: /' 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81865/exe 2024-02-05T07:36:43.901 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd on pid 82075 2024-02-05T07:36:43.905 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82075 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd on pid 82075' 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82075 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.906 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd_pp 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd_pp' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.911 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.912 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd_pp 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.913 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_snapshots_pp.xml 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.914 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81990/exe 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.915 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.916 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_stat_pp.xml 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd_pp 2024-02-05T07:36:43.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82127 2024-02-05T07:36:43.919 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd_pp on pid 82127 2024-02-05T07:36:43.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd_pp on pid 82127' 2024-02-05T07:36:43.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82127 2024-02-05T07:36:43.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.922 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service on pid 82153 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82153 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service on pid 82153' 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82153 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service_pp 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.926 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service_pp' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82016/exe 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.931 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify.xml 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-05T07:36:43.932 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service_pp 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.933 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.934 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.935 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.936 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.937 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service_pp on pid 82201 2024-02-05T07:36:43.946 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service_pp 2024-02-05T07:36:43.946 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82201 2024-02-05T07:36:43.946 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service_pp on pid 82201' 2024-02-05T07:36:43.946 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82201 2024-02-05T07:36:43.946 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.946 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_write_operations 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-05T07:36:43.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_write_operations' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_write_operations 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.952 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.953 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.954 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.955 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82028/exe 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_watch_notify_pp.xml 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82075/exe 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.962 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.963 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.963 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.963 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-05T07:36:43.963 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd.xml 2024-02-05T07:36:43.963 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_write_operations on pid 82281 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_write_operations 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82281 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_write_operations on pid 82281' 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82281 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82127/exe 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.969 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.970 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cmd_pp.xml 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.971 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-05T07:36:43.972 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd: /' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.973 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82153/exe 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_read_operations 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service: /' 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.975 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_read_operations' 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service.log 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service.xml 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82201/exe 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_read_operations 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.978 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_service_pp.xml 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.980 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_read_operations on pid 82349 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_read_operations 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82349 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_read_operations on pid 82349' 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82349 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cls_remote_reads 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cls_remote_reads' 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-05T07:36:43.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cls_remote_reads 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:43.983 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cls_remote_reads on pid 82356 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cls_remote_reads 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82356 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cls_remote_reads on pid 82356' 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82356 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s list_parallel 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' list_parallel' 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:43.986 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo list_parallel 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.987 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82281/exe 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.988 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:43.990 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_write_operations.xml 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:43.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:43.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:43.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:43.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.994 INFO:tasks.workunit.client.0.smithi043.stdout:test list_parallel on pid 82383 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=list_parallel 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82383 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test list_parallel on pid 82383' 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82383 2024-02-05T07:36:43.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:43.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s open_pools_parallel 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:43.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' open_pools_parallel' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:43.999 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo open_pools_parallel 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.001 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=open_pools_parallel 2024-02-05T07:36:44.002 INFO:tasks.workunit.client.0.smithi043.stdout:test open_pools_parallel on pid 82410 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82410 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test open_pools_parallel on pid 82410' 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82410 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.006 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s delete_pools_parallel 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' delete_pools_parallel' 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.007 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82356/exe 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.008 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_cls_remote_reads.xml 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.009 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.010 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.011 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-02-05T07:36:44.013 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo delete_pools_parallel 2024-02-05T07:36:44.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-02-05T07:36:44.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=delete_pools_parallel 2024-02-05T07:36:44.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82457 2024-02-05T07:36:44.016 INFO:tasks.workunit.client.0.smithi043.stdout:test delete_pools_parallel on pid 82457 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test delete_pools_parallel on pid 82457' 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82457 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.023 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82349/exe 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.027 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.028 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.029 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-api_c_read_operations.xml 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s cls 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' cls' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.030 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo cls 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.032 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.033 INFO:tasks.workunit.client.0.smithi043.stdout:test cls on pid 82495 2024-02-05T07:36:44.037 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=cls 2024-02-05T07:36:44.037 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.037 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82495 2024-02-05T07:36:44.037 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test cls on pid 82495' 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82495 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s cmd 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' cmd' 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.038 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.039 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.040 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82383/exe 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82410/exe 2024-02-05T07:36:44.041 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.042 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ list_parallel: /' 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo cmd 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-list_parallel.xml 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-open_pools_parallel.xml 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=cmd 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82557 2024-02-05T07:36:44.047 INFO:tasks.workunit.client.0.smithi043.stdout:test cmd on pid 82557 2024-02-05T07:36:44.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test cmd on pid 82557' 2024-02-05T07:36:44.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82557 2024-02-05T07:36:44.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.051 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.051 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s handler_error 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.052 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' handler_error' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.053 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo handler_error 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82457/exe 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.056 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.057 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.057 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.057 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.057 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/gtest_xml_report-delete_pools_parallel.xml 2024-02-05T07:36:44.057 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-05T07:36:44.057 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=handler_error 2024-02-05T07:36:44.057 INFO:tasks.workunit.client.0.smithi043.stdout:test handler_error on pid 82614 2024-02-05T07:36:44.060 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82614 2024-02-05T07:36:44.060 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-05T07:36:44.060 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test handler_error on pid 82614' 2024-02-05T07:36:44.060 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82614 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s io 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' io' 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo io 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.062 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.063 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.064 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82495/exe 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_cls.log 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ cls: /' 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.069 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_cls 2024-02-05T07:36:44.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.069 INFO:tasks.workunit.client.0.smithi043.stdout:test io on pid 82671 2024-02-05T07:36:44.071 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=io 2024-02-05T07:36:44.071 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82671 2024-02-05T07:36:44.071 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test io on pid 82671' 2024-02-05T07:36:44.071 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82671 2024-02-05T07:36:44.071 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s ec_io 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' ec_io' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo ec_io 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.073 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.074 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.075 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.076 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=ec_io 2024-02-05T07:36:44.077 INFO:tasks.workunit.client.0.smithi043.stdout:test ec_io on pid 82720 2024-02-05T07:36:44.080 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82720 2024-02-05T07:36:44.080 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test ec_io on pid 82720' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82720 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.081 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s list 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' list' 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.082 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.083 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.084 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.085 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.086 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82557/exe 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.087 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.088 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_cmd 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.089 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_cmd.log 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ cmd: /' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo list 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.090 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.091 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.091 INFO:tasks.workunit.client.0.smithi043.stdout:test list on pid 82773 2024-02-05T07:36:44.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=list 2024-02-05T07:36:44.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82773 2024-02-05T07:36:44.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test list on pid 82773' 2024-02-05T07:36:44.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82773 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82614/exe 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s ec_list 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' ec_list' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ handler_error: /' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo ec_list 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_handler_error 2024-02-05T07:36:44.100 INFO:tasks.workunit.client.0.smithi043.stdout:test ec_list on pid 82788 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=ec_list 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82788 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test ec_list on pid 82788' 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82788 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.102 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_handler_error.log 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s misc 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' misc' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo misc 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.104 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=misc 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82806 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-02-05T07:36:44.106 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test misc on pid 82806' 2024-02-05T07:36:44.107 INFO:tasks.workunit.client.0.smithi043.stdout:test misc on pid 82806 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82806 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s pool 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' pool' 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo pool 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82671/exe 2024-02-05T07:36:44.112 INFO:tasks.workunit.client.0.smithi043.stdout:test pool on pid 82826 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=pool 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82826 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test pool on pid 82826' 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82826 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s read_operations 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.118 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' read_operations' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo read_operations 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_io.log 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_io 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ io: /' 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.124 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.125 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.126 INFO:tasks.workunit.client.0.smithi043.stdout:test read_operations on pid 82873 2024-02-05T07:36:44.127 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.127 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.127 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.127 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.127 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.127 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.127 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=read_operations 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.128 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82873 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test read_operations on pid 82873' 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82873 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.129 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s snapshots 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' snapshots' 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.130 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.131 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.132 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.133 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo snapshots 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82720/exe 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.134 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.135 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.136 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.136 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.136 INFO:tasks.workunit.client.0.smithi043.stdout:test snapshots on pid 82898 2024-02-05T07:36:44.138 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=snapshots 2024-02-05T07:36:44.138 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82898 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test snapshots on pid 82898' 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82773/exe 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82898 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s watch_notify 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' watch_notify' 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo watch_notify 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.139 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.140 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.141 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.142 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_list.log 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_list 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ list: /' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.143 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ ec_io: /' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_ec_io.log 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_ec_io 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.144 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.145 INFO:tasks.workunit.client.0.smithi043.stdout:test watch_notify on pid 82926 2024-02-05T07:36:44.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=watch_notify 2024-02-05T07:36:44.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82926 2024-02-05T07:36:44.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test watch_notify on pid 82926' 2024-02-05T07:36:44.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82926 2024-02-05T07:36:44.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.150 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.151 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.152 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.153 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.154 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.155 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.156 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.157 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.158 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s write_operations 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' write_operations' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.159 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.160 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.161 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.162 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo write_operations 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82826/exe 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82806/exe 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.163 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.164 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82788/exe 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.165 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ pool: /' 2024-02-05T07:36:44.166 INFO:tasks.workunit.client.0.smithi043.stdout:test write_operations on pid 82976 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=write_operations 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_pool 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82976 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test write_operations on pid 82976' 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82976 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ ret=0 2024-02-05T07:36:44.177 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81815 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81815 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_misc 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_misc.log 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.178 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_pool.log 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_ec_list 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.179 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.180 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.181 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.182 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.183 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.184 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.185 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ misc: /' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.186 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.187 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_ec_list.log 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82898/exe 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ ec_list: /' 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.188 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.189 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_snapshots 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.190 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_snapshots.log 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ snapshots: /' 2024-02-05T07:36:44.191 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82873/exe 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.192 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.193 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.194 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.195 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_read_operations 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ read_operations: /' 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_read_operations.log 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82926/exe 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.196 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_watch_notify 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ watch_notify: /' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-02-05T07:36:44.197 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-02-05T07:36:44.198 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-02-05T07:36:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82976/exe 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_write_operations 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_write_operations.log 2024-02-05T07:36:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ write_operations: /' 2024-02-05T07:36:45.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/569360749' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-05T07:36:45.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:45.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/569360749' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-05T07:36:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/569360749' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1175073757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2350522519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[29879]: osdmap e75: 8 total, 8 up, 8 in 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1175073757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2350522519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[41444]: osdmap e75: 8 total, 8 up, 8 in 2024-02-05T07:36:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1175073757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2350522519' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:46 smithi050 ceph-mon[33492]: osdmap e75: 8 total, 8 up, 8 in 2024-02-05T07:36:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:36:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3020703177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1255465299' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/557234704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3843338492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: osdmap e76: 8 total, 8 up, 8 in 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82977-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3020703177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1255465299' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/557234704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3843338492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: osdmap e76: 8 total, 8 up, 8 in 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82977-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3020703177' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1255465299' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/557234704' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3843338492' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: osdmap e76: 8 total, 8 up, 8 in 2024-02-05T07:36:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82977-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:51.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:36:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:36:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3302571616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1099927138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[29879]: osdmap e77: 8 total, 8 up, 8 in 2024-02-05T07:36:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3302571616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1099927138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[41444]: osdmap e77: 8 total, 8 up, 8 in 2024-02-05T07:36:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3302571616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1099927138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:51 smithi050 ceph-mon[33492]: osdmap e77: 8 total, 8 up, 8 in 2024-02-05T07:36:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82977-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/123704972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2303591468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2708192752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1565303852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2921790509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1472156630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4040986189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: osdmap e78: 8 total, 8 up, 8 in 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82977-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82977-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/123704972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2303591468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2708192752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1565303852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2921790509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1472156630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4040986189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: osdmap e78: 8 total, 8 up, 8 in 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82977-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/43743506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4061021414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2027663683' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:36:53.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[29879]: osdmap e79: 8 total, 8 up, 8 in 2024-02-05T07:36:53.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:53.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/43743506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4061021414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2027663683' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:36:53.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:53 smithi043 ceph-mon[41444]: osdmap e79: 8 total, 8 up, 8 in 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82977-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81692-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/123704972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2303591468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2708192752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1565303852' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2921790509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1472156630' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4040986189' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: osdmap e78: 8 total, 8 up, 8 in 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82977-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81716-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82912-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81996-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/43743506' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4061021414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2027663683' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:36:53.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:53 smithi050 ceph-mon[33492]: osdmap e79: 8 total, 8 up, 8 in 2024-02-05T07:36:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: pgmap v68: 324 pgs: 181 creating+peering, 132 active+clean, 11 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: pgmap v70: 452 pgs: 181 creating+peering, 132 active+clean, 139 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: pgmap v71: 452 pgs: 181 creating+peering, 132 active+clean, 139 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: pgmap v73: 932 pgs: 731 creating+peering, 152 active+clean, 49 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-05T07:36:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: Health check failed: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: osdmap e80: 8 total, 8 up, 8 in 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/618490397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: pgmap v68: 324 pgs: 181 creating+peering, 132 active+clean, 11 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: pgmap v70: 452 pgs: 181 creating+peering, 132 active+clean, 139 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: pgmap v71: 452 pgs: 181 creating+peering, 132 active+clean, 139 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: pgmap v73: 932 pgs: 731 creating+peering, 152 active+clean, 49 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: Health check failed: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: osdmap e80: 8 total, 8 up, 8 in 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/618490397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:54.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: pgmap v68: 324 pgs: 181 creating+peering, 132 active+clean, 11 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:36:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: pgmap v70: 452 pgs: 181 creating+peering, 132 active+clean, 139 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: pgmap v71: 452 pgs: 181 creating+peering, 132 active+clean, 139 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:36:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:36:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: pgmap v73: 932 pgs: 731 creating+peering, 152 active+clean, 49 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-02-05T07:36:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: Health check failed: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:36:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: osdmap e80: 8 total, 8 up, 8 in 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/618490397' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:54.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment set-up. 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (3 ms) 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-05T07:36:54.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (8 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (48 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (25 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (18 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (3 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (4 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (3 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (4 ms) 2024-02-05T07:36:54.408 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (3 ms) 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (4 ms) 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados (124 ms total) 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment tear-down 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (10679 ms total) 2024-02-05T07:36:54.409 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment set-up. 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjects (457 ms) 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (34 ms) 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-02-05T07:36:54.736 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-02-05T07:36:54.737 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (80 ms) 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-02-05T07:36:54.738 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (59 ms) 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: x cursor=MIN 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=1 cursor=15:02547ec2:::1:head 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=10 cursor=15:52ea6a34:::10:head 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=13 cursor=15:566253c9:::13:head 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=7 cursor=15:5c6b0b28:::7:head 2024-02-05T07:36:54.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=14 cursor=15:62a1935d:::14:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=0 cursor=15:6cac518f:::0:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=15 cursor=15:863748b0:::15:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=11 cursor=15:89d3ae78:::11:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=5 cursor=15:b29083e3:::5:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=8 cursor=15:bd63b0f1:::8:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=6 cursor=15:c4fdafeb:::6:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=3 cursor=15:cfc208b3:::3:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=4 cursor=15:d83876eb:::4:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=12 cursor=15:de5d7c5f:::12:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=9 cursor=15:e960b815:::9:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=2 cursor=15:f905c69b:::2:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=1 cursor=15:02547ec2:::1:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:02547ec2:::1:head 2024-02-05T07:36:54.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:02547ec2:::1:head -> 1 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=10 cursor=15:52ea6a34:::10:head 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:52ea6a34:::10:head 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:52ea6a34:::10:head -> 10 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=13 cursor=15:566253c9:::13:head 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:566253c9:::13:head 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:566253c9:::13:head -> 13 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=7 cursor=15:5c6b0b28:::7:head 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:5c6b0b28:::7:head 2024-02-05T07:36:54.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:5c6b0b28:::7:head -> 7 2024-02-05T07:36:54.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=14 cursor=15:62a1935d:::14:head 2024-02-05T07:36:54.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:62a1935d:::14:head 2024-02-05T07:36:54.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:62a1935d:::14:head -> 14 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=0 cursor=15:6cac518f:::0:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:6cac518f:::0:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:6cac518f:::0:head -> 0 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=15 cursor=15:863748b0:::15:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:863748b0:::15:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:863748b0:::15:head -> 15 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=11 cursor=15:89d3ae78:::11:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:89d3ae78:::11:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:89d3ae78:::11:head -> 11 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=5 cursor=15:b29083e3:::5:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:b29083e3:::5:head 2024-02-05T07:36:54.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:b29083e3:::5:head -> 5 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=8 cursor=15:bd63b0f1:::8:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:bd63b0f1:::8:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:bd63b0f1:::8:head -> 8 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=6 cursor=15:c4fdafeb:::6:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:c4fdafeb:::6:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:c4fdafeb:::6:head -> 6 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=3 cursor=15:cfc208b3:::3:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:cfc208b3:::3:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:cfc208b3:::3:head -> 3 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=4 cursor=15:d83876eb:::4:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:d83876eb:::4:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:d83876eb:::4:head -> 4 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=12 cursor=15:de5d7c5f:::12:head 2024-02-05T07:36:54.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:de5d7c5f:::12:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:de5d7c5f:::12:head -> 12 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=9 cursor=15:e960b815:::9:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:e960b815:::9:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:e960b815:::9:head -> 9 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=2 cursor=15:f905c69b:::2:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:f905c69b:::2:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:f905c69b:::2:head -> 2 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:566253c9:::13:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:566253c9:::13:head expected=15:566253c9:::13:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:566253c9:::13:head -> 13 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=13 expected=13 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:cfc208b3:::3:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:cfc208b3:::3:head expected=15:cfc208b3:::3:head 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:cfc208b3:::3:head -> 3 2024-02-05T07:36:54.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=3 expected=3 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:c4fdafeb:::6:head 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:c4fdafeb:::6:head expected=15:c4fdafeb:::6:head 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:c4fdafeb:::6:head -> 6 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=6 expected=6 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:bd63b0f1:::8:head 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:bd63b0f1:::8:head expected=15:bd63b0f1:::8:head 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:bd63b0f1:::8:head -> 8 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=8 expected=8 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:b29083e3:::5:head 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:b29083e3:::5:head expected=15:b29083e3:::5:head 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:b29083e3:::5:head -> 5 2024-02-05T07:36:54.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=5 expected=5 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:89d3ae78:::11:head 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:89d3ae78:::11:head expected=15:89d3ae78:::11:head 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:89d3ae78:::11:head -> 11 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=11 expected=11 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:5c6b0b28:::7:head 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:5c6b0b28:::7:head expected=15:5c6b0b28:::7:head 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:5c6b0b28:::7:head -> 7 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=7 expected=7 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:863748b0:::15:head 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:863748b0:::15:head expected=15:863748b0:::15:head 2024-02-05T07:36:54.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:863748b0:::15:head -> 15 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: pgmap v76: 1252 pgs: 701 creating+peering, 152 active+clean, 399 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82977-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: pgmap v76: 1252 pgs: 701 creating+peering, 152 active+clean, 399 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82977-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4267279926' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: osdmap e81: 8 total, 8 up, 8 in 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:55.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:55.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:55.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4267279926' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:36:55.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: osdmap e81: 8 total, 8 up, 8 in 2024-02-05T07:36:55.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:55.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: pgmap v76: 1252 pgs: 701 creating+peering, 152 active+clean, 399 unknown; 582 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82977-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81747-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81795-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82170-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81843-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81789-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82081-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81993-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82431-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82052-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82119-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82468-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82274-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82292-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81739-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4267279926' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: osdmap e81: 8 total, 8 up, 8 in 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:36:55.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:56.164 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : ent list_parallel: process_1_[82611]: starting. 2024-02-05T07:36:56.164 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82611]: creating pool ceph_test_rados_list_parallel.smithi043-82509 2024-02-05T07:36:56.164 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82611]: created object 0... 2024-02-05T07:36:56.164 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82611]: created object 25... 2024-02-05T07:36:56.164 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82611]: created object 49... 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82611]: finishing. 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82611]: shutting down. 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82613]: starting. 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82613]: listing objects. 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82613]: listed object 0... 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82613]: listed object 25... 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82613]: saw 50 objects 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82613]: shutting down. 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83741]: starting. 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83741]: creating pool ceph_test_rados_list_parallel.smithi043-82509 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83741]: created object 0... 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83741]: created object 25... 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83741]: created object 49... 2024-02-05T07:36:56.165 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83741]: finishing. 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83741]: shutting down. 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83743]: starting. 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83743]: removed 25 objects... 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83743]: removed half of the objects 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83743]: removed 50 objects... 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83743]: removed 50 objects 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83743]: shutting down. 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83742]: starting. 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83742]: listing objects. 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83742]: saw 0 objects 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83742]: shutting down. 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.166 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83796]: starting. 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83796]: creating pool ceph_test_rados_list_parallel.smithi043-82509 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83796]: created object 0... 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83796]: created object 25... 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83796]: created object 49... 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83796]: finishing. 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83796]: shutting down. 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83797]: starting. 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83797]: listing objects. 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83797]: listed object 0... 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83797]: listed object 25... 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83797]: listed object 50... 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83797]: saw 56 objects 2024-02-05T07:36:56.167 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83797]: shutting down. 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83798]: starting. 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83798]: added 25 objects... 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83798]: added half of the objects 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83798]: added 50 objects... 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83798]: added 50 objects 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83798]: shutting down. 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.168 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83876]: starting. 2024-02-05T07:36:56.490 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83876]: creating pool ceph_t handler_error: Running main() from gmock_main.cc 2024-02-05T07:36:56.490 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-02-05T07:36:56.490 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] Global test environment set-up. 2024-02-05T07:36:56.490 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-02-05T07:36:56.491 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-02-05T07:36:56.491 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (12458 ms) 2024-02-05T07:36:56.491 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] 1 test from neocls_handler_error (12458 ms total) 2024-02-05T07:36:56.491 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: 2024-02-05T07:36:56.491 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] Global test environment tear-down 2024-02-05T07:36:56.491 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [==========] 1 test from 1 test suite ran. (12458 ms total) 2024-02-05T07:36:56.491 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ PASSED ] 1 test. 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout:est_rados_list_parallel.smithi043-82509 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83876]: created object 0... 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83876]: created object 25... 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83876]: created object 49... 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83876]: finishing. 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83876]: shutting down. 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: starting. 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: listing objects. 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: listed object 0... 2024-02-05T07:36:56.574 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: listed object 25... 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: listed object 50... 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: listed object 75... 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: saw 99 objects 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83877]: shutting down. 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83878]: starting. 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83878]: added 25 objects... 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83878]: added half of the objects 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83878]: added 50 objects... 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83878]: added 50 objects 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83878]: shutting down. 2024-02-05T07:36:56.575 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83880]: starting. 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83880]: removed 25 objects... 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83880]: removed half of the objects 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83880]: removed 50 objects... 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83880]: removed 50 objects 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83880]: shutting down. 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83879]: starting. 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83879]: added 25 objects... 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83879]: added half of the objects 2024-02-05T07:36:56.576 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83879]: added 50 objects... 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83879]: added 50 objects 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83879]: shutting down. 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83958]: starting. 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83958]: creating pool ceph_test_rados_list_parallel.smithi043-82509 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83958]: created object 0... 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83958]: created object 25... 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83958]: created object 49... 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83958]: finishing. 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83958]: shutting down. 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.577 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.578 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.578 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:56.578 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: starting. 2024-02-05T07:36:56.578 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: listing objects. 2024-02-05T07:36:56.578 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: listed object 0... 2024-02-05T07:36:56.578 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: listed object 25... 2024-02-05T07:36:56.578 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: listed object 50... 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/110240231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[29879]: osdmap e82: 8 total, 8 up, 8 in 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/110240231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[41444]: osdmap e82: 8 total, 8 up, 8 in 2024-02-05T07:36:56.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82912-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:36:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/110240231' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:56 smithi050 ceph-mon[33492]: osdmap e82: 8 total, 8 up, 8 in 2024-02-05T07:36:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:57.509 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: listed object api_cmd_pp: Running main() from gmock_main.cc 2024-02-05T07:36:57.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:36:57.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-05T07:36:57.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-05T07:36:57.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-05T07:36:57.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (48 ms) 2024-02-05T07:36:57.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (51 ms) 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (13547 ms) 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (13646 ms total) 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (13646 ms total) 2024-02-05T07:36:57.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-05T07:36:57.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-02-05T07:36:57.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-02-05T07:36:57.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-05T07:36:57.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-02-05T07:36:57.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (408 ms) 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (10 ms) 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (9 ms) 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (28 ms) 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (8 ms) 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-05T07:36:57.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (4 ms) 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (6 ms) 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (14 ms) 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-05T07:36:57.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (9 ms) 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (7 ms) 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (5 ms) 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (537 ms total) 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (13576 ms total) 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-02-05T07:36:57.523 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-02-05T07:36:57.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-02-05T07:36:57.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (13589 ms) 2024-02-05T07:36:57.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (13589 ms total) 2024-02-05T07:36:57.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: 2024-02-05T07:36:57.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-02-05T07:36:57.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (13589 ms total) 2024-02-05T07:36:57.524 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-02-05T07:36:57.586 INFO:tasks.workunit.client.0.smithi043.stdout: 75... 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: listed object 100... 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: listed object 125... 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: saw 150 objects 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83959]: shutting down. 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83960]: starting. 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83960]: added 25 objects... 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83960]: added half of the objects 2024-02-05T07:36:57.587 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83960]: added 50 objects... 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83960]: added 50 objects 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83960]: shutting down. 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-02-05T07:36:57.588 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******* SUCCESS ********** 2024-02-05T07:36:57.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: pgmap v79: 1236 pgs: 102 creating+peering, 1117 active+clean, 17 unknown; 626 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 92 KiB/s wr, 564 op/s 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:36:57.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: osdmap e83: 8 total, 8 up, 8 in 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: pgmap v79: 1236 pgs: 102 creating+peering, 1117 active+clean, 17 unknown; 626 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 92 KiB/s wr, 564 op/s 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:36:57.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: osdmap e83: 8 total, 8 up, 8 in 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:57.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:57.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:57.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: pgmap v79: 1236 pgs: 102 creating+peering, 1117 active+clean, 17 unknown; 626 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 33 KiB/s rd, 92 KiB/s wr, 564 op/s 2024-02-05T07:36:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:36:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:36:57.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: osdmap e83: 8 total, 8 up, 8 in 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:57.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.168 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (53 ms) 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (43 ms) 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (13564 ms) 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.477630+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82244-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:36:59.169 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.674698+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.675463+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.675959+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.676486+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.676984+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.677493+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.677946+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.678423+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.678870+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.679455+0000 mon.b [INF] from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.769098+0000 mon.a [INF] from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:56.770460+0000 mon.a [INF] from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.034943+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.035129+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.170 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.480602+0000 mon.a [INF] from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.480687+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.480764+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.491847+0000 mon.b [INF] from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.493372+0000 mon.a [INF] from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.508788+0000 mon.c [INF] from='client.? 172.21.15.43:0/4174055166' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.603281+0000 mon.b [INF] from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.607314+0000 mon.c [INF] from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.610049+0000 client.admin [INF] onexx 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.625882+0000 mon.b [INF] from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.646966+0000 mon.c [INF] from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.649785+0000 mon.b [INF] from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.654243+0000 mon.b [INF] from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.873722+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.171 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:57.876994+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.172 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:58.131016+0000 mon.a [INF] from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:36:59.172 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:58.131306+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.178 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:58.131592+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", cmd: Running main() from gmock_main.cc 2024-02-05T07:36:59.178 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:36:59.178 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] Global test environment set-up. 2024-02-05T07:36:59.178 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-02-05T07:36:59.178 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-02-05T07:36:59.178 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (9276 ms) 2024-02-05T07:36:59.178 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-02-05T07:36:59.179 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (2538 ms) 2024-02-05T07:36:59.179 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-02-05T07:36:59.179 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (3338 ms) 2024-02-05T07:36:59.179 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] 3 tests from NeoRadosCmd (15152 ms total) 2024-02-05T07:36:59.179 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: 2024-02-05T07:36:59.179 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] Global test environment tear-down 2024-02-05T07:36:59.179 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [==========] 3 tests from 1 test suite ran. (15152 ms total) 2024-02-05T07:36:59.180 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ PASSED ] 3 tests. 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:36:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: 19.0 deep-scrub starts 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: 19.0 deep-scrub ok 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: 19.7 deep-scrub starts 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: 19.7 deep-scrub ok 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: osdmap e84: 8 total, 8 up, 8 in 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4174055166' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: onexx 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: 19.0 deep-scrub starts 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: 19.0 deep-scrub ok 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: 19.7 deep-scrub starts 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: 19.7 deep-scrub ok 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: osdmap e84: 8 total, 8 up, 8 in 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:36:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4174055166' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: onexx 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:36:59.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:36:59.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:36:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: 19.0 deep-scrub starts 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: 19.0 deep-scrub ok 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: 19.7 deep-scrub starts 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: 19.7 deep-scrub ok 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81692-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: osdmap e84: 8 total, 8 up, 8 in 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4174055166' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: onexx 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:36:59.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:36:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:00.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: 19.2 deep-scrub starts 2024-02-05T07:37:00.752 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: 19.2 deep-scrub ok 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: osdmap e85: 8 total, 8 up, 8 in 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-05T07:37:00.753 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:00.754 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: 19.2 deep-scrub starts 2024-02-05T07:37:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: 19.2 deep-scrub ok 2024-02-05T07:37:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: osdmap e85: 8 total, 8 up, 8 in 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: 19.2 deep-scrub starts 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: 19.2 deep-scrub ok 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:00.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2438408192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: osdmap e85: 8 total, 8 up, 8 in 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82977-1"}]: dispatch 2024-02-05T07:37:00.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:00.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:00.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:00.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:00.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:00.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:01.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:37:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:37:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.3 deep-scrub starts 2024-02-05T07:37:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.3 deep-scrub ok 2024-02-05T07:37:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.6 deep-scrub starts 2024-02-05T07:37:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.6 deep-scrub ok 2024-02-05T07:37:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.1 deep-scrub starts 2024-02-05T07:37:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.1 deep-scrub ok 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: pgmap v82: 652 pgs: 66 creating+peering, 445 active+clean, 141 unknown; 586 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 21 KiB/s wr, 439 op/s 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.9 deep-scrub starts 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.9 deep-scrub ok 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.5 deep-scrub starts 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.5 deep-scrub ok 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.4 deep-scrub starts 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.4 deep-scrub ok 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: twoxx 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.8 deep-scrub starts 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: 19.8 deep-scrub ok 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: pgmap v84: 780 pgs: 66 creating+peering, 445 active+clean, 269 unknown; 586 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 19 KiB/s wr, 400 op/s 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:01.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: osdmap e86: 8 total, 8 up, 8 in 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81843-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.3 deep-scrub starts 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.3 deep-scrub ok 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.6 deep-scrub starts 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.6 deep-scrub ok 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.1 deep-scrub starts 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.1 deep-scrub ok 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: pgmap v82: 652 pgs: 66 creating+peering, 445 active+clean, 141 unknown; 586 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 21 KiB/s wr, 439 op/s 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.9 deep-scrub starts 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.9 deep-scrub ok 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.5 deep-scrub starts 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.5 deep-scrub ok 2024-02-05T07:37:01.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.4 deep-scrub starts 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.4 deep-scrub ok 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: twoxx 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.8 deep-scrub starts 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: 19.8 deep-scrub ok 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: pgmap v84: 780 pgs: 66 creating+peering, 445 active+clean, 269 unknown; 586 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 19 KiB/s wr, 400 op/s 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: osdmap e86: 8 total, 8 up, 8 in 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81843-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:01.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:01.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/135776579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:01.820 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/135776579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: osdmap e87: 8 total, 8 up, 8 in 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-4", "overlaypool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:01.821 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: osdmap e87: 8 total, 8 up, 8 in 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-4", "overlaypool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.822 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.3 deep-scrub starts 2024-02-05T07:37:01.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.3 deep-scrub ok 2024-02-05T07:37:01.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.6 deep-scrub starts 2024-02-05T07:37:01.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.6 deep-scrub ok 2024-02-05T07:37:01.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.1 deep-scrub starts 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.1 deep-scrub ok 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: pgmap v82: 652 pgs: 66 creating+peering, 445 active+clean, 141 unknown; 586 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 21 KiB/s wr, 439 op/s 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.9 deep-scrub starts 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.9 deep-scrub ok 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.5 deep-scrub starts 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.5 deep-scrub ok 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.4 deep-scrub starts 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.4 deep-scrub ok 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: twoxx 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.8 deep-scrub starts 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: 19.8 deep-scrub ok 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: pgmap v84: 780 pgs: 66 creating+peering, 445 active+clean, 269 unknown; 586 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 19 KiB/s wr, 400 op/s 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:01.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1730123213' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82977-1"}]': finished 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: osdmap e86: 8 total, 8 up, 8 in 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81843-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82912-1"}]': finished 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81716-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82977-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/602056817' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/135776579' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:01.826 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: osdmap e87: 8 total, 8 up, 8 in 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-4", "overlaypool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:01.827 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: Running main() from gmock_main.cc 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] Global test environment set-up. 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ OK ] NeoRadosCls.DNE (13508 ms) 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-02-05T07:37:02.370 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (4856 ms) 2024-02-05T07:37:02.371 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] 2 tests from NeoRadosCls (18364 ms total) 2024-02-05T07:37:02.371 INFO:tasks.workunit.client.0.smithi043.stdout: cls: 2024-02-05T07:37:02.371 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] Global test environment tear-down 2024-02-05T07:37:02.371 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [==========] 2 tests from 1 test suite ran. (18364 ms total) 2024-02-05T07:37:02.371 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ PASSED ] 2 tests. 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81843-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-4", "overlaypool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:03.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81843-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-4", "overlaypool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: osdmap e88: 8 total, 8 up, 8 in 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2798617414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/461606154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: osdmap e88: 8 total, 8 up, 8 in 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2798617414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/461606154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:03.068 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:03.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:03.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81843-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-4", "overlaypool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: osdmap e88: 8 total, 8 up, 8 in 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2798617414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/461606154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:03.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:03.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:03.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:03.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:58.132273+0000 mon.b [INF] from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:36:58.381338+0000 mon.a [INF] from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:01.554466+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:01.555023+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:01.555558+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:01.560708+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066409+0000 mon.a [INF] from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81843-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066511+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066589+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81795-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066668+0000 mon.a [INF] from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-4", "overlaypool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066727+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81716-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066807+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:03.561 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066888+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82912-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.066973+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82052-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.067049+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.067098+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81739-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.071145+0000 mon.c [INF] from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.071842+0000 mon.b [INF] from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.071877+0000 mon.c [INF] from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.072076+0000 mon.b [INF] from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.072309+0000 mon.b [INF] from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.072519+0000 mon.b [INF] from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.364207+0000 mon.b [INF] from='client.? 172.21.15.43:0/2798617414' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.364410+0000 mon.b [INF] from='client.? 172.21.15.43:0/461606154' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.366253+0000 mon.a [INF] from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.366598+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:03.562 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.367182+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.367703+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119 delete_pools_parallel: process_1_[82728]: starting. 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82728]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-82607 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82728]: created object 0... 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82728]: created object 25... 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82728]: created object 49... 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82728]: finishing. 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82728]: shutting down. 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[82732]: starting. 2024-02-05T07:37:04.136 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[82732]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-82607 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[82732]: shutting down. 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84039]: starting. 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84039]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-82607 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84039]: created object 0... 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84039]: created object 25... 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84039]: created object 49... 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84039]: finishing. 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84039]: shutting down. 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84041]: starting. 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84041]: listing objects. 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84041]: listed object 0... 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84041]: listed object 25... 2024-02-05T07:37:04.137 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84041]: saw 50 objects 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84041]: shutting down. 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[84040]: starting. 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[84040]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-82607 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[84040]: shutting down. 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-02-05T07:37:04.138 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82602]: starting. 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82602]: creating pool ceph_test_rados_open_pools_parallel.smithi043-82514 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82602]: created object 0... 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82602]: created object 25... 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82602]: created object 49... 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82602]: finishing. 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82602]: shutting down. 2024-02-05T07:37:04.638 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82606]: starting. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82606]: rados_pool_create. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82606]: rados_ioctx_create. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82606]: shutting down. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84042]: starting. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84042]: creating pool ceph_test_rados_open_pools_parallel.smithi043-82514 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84042]: created object 0... 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84042]: created object 25... 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84042]: created object 49... 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84042]: finishing. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84042]: shutting down. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84043]: starting. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84043]: rados_pool_create. 2024-02-05T07:37:04.639 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84043]: rados_ioctx_create. 2024-02-05T07:37:04.640 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84043]: shutting down. 2024-02-05T07:37:04.640 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-02-05T07:37:04.640 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-02-05T07:37:04.640 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-05T07:37:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[29879]: pgmap v87: 900 pgs: 269 creating+peering, 503 active+clean, 128 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 36 MiB/s wr, 104 op/s 2024-02-05T07:37:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:05.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[41444]: pgmap v87: 900 pgs: 269 creating+peering, 503 active+clean, 128 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 36 MiB/s wr, 104 op/s 2024-02-05T07:37:05.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:05.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:05.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:05.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:04 smithi050 ceph-mon[33492]: pgmap v87: 900 pgs: 269 creating+peering, 503 active+clean, 128 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 11 MiB/s rd, 36 MiB/s wr, 104 op/s 2024-02-05T07:37:05.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:05.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:04 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:05.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:05.978 INFO:tasks.workunit.client.0.smithi043.stdout:-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:05.978 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.368146+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:05.978 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.371494+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:05.978 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.374384+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:05.978 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.374768+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:05.978 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.643530+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:02.643703+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.068770+0000 mon.a [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.073303+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82977-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.073379+0000 mon.a [INF] from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]': finished 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.073430+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.073511+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.073588+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.080493+0000 mon.c [INF] from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.560522+0000 mon.c [INF] from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:03.560686+0000 client.admin [INF] threexx 2024-02-05T07:37:05.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.636876+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:06.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]': finished 2024-02-05T07:37:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:37:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:06.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: threexx 2024-02-05T07:37:06.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: pgmap v90: 756 pgs: 56 creating+peering, 492 active+clean, 208 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 109 op/s 2024-02-05T07:37:06.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:06.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: osdmap e89: 8 total, 8 up, 8 in 2024-02-05T07:37:06.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:06.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:06.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:06.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:06.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:06.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:06.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:06.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:06.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: osdmap e90: 8 total, 8 up, 8 in 2024-02-05T07:37:06.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-4"}]: dispatch 2024-02-05T07:37:06.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:06.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]': finished 2024-02-05T07:37:06.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:37:06.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:06.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:06.317 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: threexx 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: pgmap v90: 756 pgs: 56 creating+peering, 492 active+clean, 208 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 109 op/s 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: osdmap e89: 8 total, 8 up, 8 in 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:06.318 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:06.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:06.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:06.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:06.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:06.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:06.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: osdmap e90: 8 total, 8 up, 8 in 2024-02-05T07:37:06.319 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-4"}]: dispatch 2024-02-05T07:37:06.320 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-4-cache", "mode": "writeback"}]': finished 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81692-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1672305025' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: threexx 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: pgmap v90: 756 pgs: 56 creating+peering, 492 active+clean, 208 unknown; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 109 op/s 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:06.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: osdmap e89: 8 total, 8 up, 8 in 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]: dispatch 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: osdmap e90: 8 total, 8 up, 8 in 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-4"}]: dispatch 2024-02-05T07:37:06.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: seed 81747 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (374 ms) 2024-02-05T07:37:06.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (24 ms) 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (4 ms) 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (7 ms) 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (6 ms) 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (3 ms) 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (3 ms) 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (7 ms) 2024-02-05T07:37:06.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (5 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (4 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (5 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (5 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (13 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (7 ms) 2024-02-05T07:37:06.886 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (4 ms) 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (5 ms) 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (487 ms total) 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (2194 ms) 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (20 ms) 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-05T07:37:06.887 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-02-05T07:37:06.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-05T07:37:06.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (3 ms) 2024-02-05T07:37:06.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-05T07:37:06.888 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-4"}]': finished 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/888590639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: osdmap e91: 8 total, 8 up, 8 in 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:37:07.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: fourxx 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: osdmap e92: 8 total, 8 up, 8 in 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2075962437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:07.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.325 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:37:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:37:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-4"}]': finished 2024-02-05T07:37:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/888590639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: osdmap e91: 8 total, 8 up, 8 in 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: fourxx 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: osdmap e92: 8 total, 8 up, 8 in 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2075962437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82244-1"}]': finished 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-4"}]': finished 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/888590639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: osdmap e91: 8 total, 8 up, 8 in 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]: dispatch 2024-02-05T07:37:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2076711079' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: fourxx 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-4", "tierpool": "test-rados-api-smithi043-81993-4-cache"}]': finished 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: osdmap e92: 8 total, 8 up, 8 in 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2075962437' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]: dispatch 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:07.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-05T07:37:07.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] watch_notify: Running main() from gmock_main.cc 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] Global test environment set-up. 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: handle_notify cookie 94891237726240 notify_id 347892350977 notifier_gid 24875 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (13399 ms) 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Trying... 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: handle_notify cookie 94891250435632 notify_id 377957122050 notifier_gid 25135 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Waiting for 3.000000000s 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Timed out. 2024-02-05T07:37:07.768 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Flushing... 2024-02-05T07:37:07.769 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Flushed... 2024-02-05T07:37:07.769 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (10246 ms) 2024-02-05T07:37:07.769 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (23645 ms total) 2024-02-05T07:37:07.769 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: 2024-02-05T07:37:07.769 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] Global test environment tear-down 2024-02-05T07:37:07.769 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (23645 ms total) 2024-02-05T07:37:07.769 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ PASSED ] 2 tests. 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: seed 81843 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (395 ms) 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (6 ms) 2024-02-05T07:37:07.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1017 ms) 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1007 ms) 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (8 ms) 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (30 ms) 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (54 ms) 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2521 ms total) 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-05T07:37:07.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (851 ms) 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (7 ms) 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1004 ms) 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1006 ms) 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (4 ms) 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (4 ms) 2024-02-05T07:37:07.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-05T07:37:07.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-02-05T07:37:07.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (2883 ms total) 2024-02-05T07:37:07.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-02-05T07:37:07.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-05T07:37:07.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (24084 ms total) 2024-02-05T07:37:07.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-05T07:37:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: pgmap v92: 804 pgs: 120 creating+peering, 684 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-02-05T07:37:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-05T07:37:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: osdmap e93: 8 total, 8 up, 8 in 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: pgmap v92: 804 pgs: 120 creating+peering, 684 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: osdmap e93: 8 total, 8 up, 8 in 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2019800219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2019800219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.067 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.068 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: pgmap v92: 804 pgs: 120 creating+peering, 684 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 3.4 KiB/s rd, 5.3 KiB/s wr, 11 op/s 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1624401629' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81843-10"}]': finished 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81692-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: osdmap e93: 8 total, 8 up, 8 in 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2019800219' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:09.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.075 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.647242+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81795-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:09.075 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.647344+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82052-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.647440+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82119-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.647532+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81739-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.647621+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81747-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.647718+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82912-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.924584+0000 mon.a [INF] from='client.? 172.21.15.43:0/2473592053' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-4"}]: dispatch 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-02-05T07:37:04.929610+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (11560 ms) 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (25220 ms total) 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (25221 ms total) 2024-02-05T07:37:09.076 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-05T07:37:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:37:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: Running main() from gmock_main.cc 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] Global test environment set-up. 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (1319 ms) 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-02-05T07:37:09.903 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (30 ms) 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (1173 ms) 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.Checksum (2 ms) 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (3 ms) 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (5 ms) 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (3 ms) 2024-02-05T07:37:09.904 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (3 ms) 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (5 ms) 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RmXattr (19 ms) 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrIter (5 ms) 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo (2575 ms total) 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-05T07:37:09.905 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (2181 ms) 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (10 ms) 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (5 ms) 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (5 ms) 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-05T07:37:09.906 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC (2231 ms total) 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [- ec_io: Running main() from gmock_main.cc 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-02-05T07:37:09.907 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] Global test environment set-up. 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (16919 ms) 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (8912 ms) 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (25831 ms total) 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] Global test environment tear-down 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (25831 ms total) 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ PASSED ] 2 tests. 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout:---------] Global test environment tear-down 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] 24 tests from 2 test suites ran. (26239 ms total) 2024-02-05T07:37:09.908 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ PASSED ] 24 tests. 2024-02-05T07:37:09.910 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-05T07:37:09.910 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-05T07:37:09.910 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-05T07:37:09.910 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-05T07:37:09.910 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: seed 82119 2024-02-05T07:37:09.910 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (351 ms) 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (42 ms) 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (4 ms) 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (2 ms) 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (18 ms) 2024-02-05T07:37:09.911 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (417 ms total) 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (2193 ms) 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (8 ms) 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (2205 ms total) 2024-02-05T07:37:09.912 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-02-05T07:37:09.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-05T07:37:09.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (26101 ms total) 2024-02-05T07:37:09.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-05T07:37:09.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: Running main() from gmock_main.cc 2024-02-05T07:37:09.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-02-05T07:37:09.913 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment set-up. 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.Stat (331 ms) 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (11 ms) 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.StatNS (64 ms) 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (5 ms) 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 5 tests from LibRadosStat (411 ms total) 2024-02-05T07:37:09.914 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (2167 ms) 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (37 ms) 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (2207 ms total) 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment tear-down 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (26142 ms total) 2024-02-05T07:37:09.915 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ PASSED ] 9 tests. 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: Running main() from gmock_main.cc 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] Global test environment set-up. 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] 3 tests from NeoradosList 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjects (13436 ms) 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjectsNS (7129 ms) 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjectsMany (5307 ms) 2024-02-05T07:37:09.944 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] 3 tests from NeoradosList (25872 ms total) 2024-02-05T07:37:09.945 INFO:tasks.workunit.client.0.smithi043.stdout: list: 2024-02-05T07:37:09.945 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] Global test environment tear-down 2024-02-05T07:37:09.945 INFO:tasks.workunit.client.0.smithi043.stdout: list: [==========] 3 tests from 1 test suite ran. (25873 ms total) 2024-02-05T07:37:09.945 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ PASSED ] 3 tests. 2024-02-05T07:37:09.945 INFO:tasks.workunit.client.0.smithi043.stdout: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-05T07:37:09.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (3 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (3 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (3 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (4 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (3 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (4 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (9 ms) 2024-02-05T07:37:09.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (4 ms) 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (2 ms) 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (3 ms) 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (2268 ms total) 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (26272 ms total) 2024-02-05T07:37:09.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: pgmap v95: 924 pgs: 256 unknown, 88 creating+peering, 580 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82244-6", "pg_num": 4}]: dispatch 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82244-6", "pg_num": 4}]': finished 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: osdmap e94: 8 total, 8 up, 8 in 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/694850954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: pgmap v95: 924 pgs: 256 unknown, 88 creating+peering, 580 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82244-6", "pg_num": 4}]: dispatch 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:09.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82244-6", "pg_num": 4}]': finished 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:37:09.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: osdmap e94: 8 total, 8 up, 8 in 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/694850954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:09.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: pgmap v95: 924 pgs: 256 unknown, 88 creating+peering, 580 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:37:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82244-6", "pg_num": 4}]: dispatch 2024-02-05T07:37:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:10.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82244-6", "pg_num": 4}]': finished 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/952951264' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1086106537' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4039949237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3344212509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1478539845' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2720355094' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: osdmap e94: 8 total, 8 up, 8 in 2024-02-05T07:37:10.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/694850954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]: dispatch 2024-02-05T07:37:10.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:11.069 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:37:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:37:11.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:11.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: osdmap e95: 8 total, 8 up, 8 in 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "overlaypool": "test-rados-api-smithi043-82244-6"}]: dispatch 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: osdmap e95: 8 total, 8 up, 8 in 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "overlaypool": "test-rados-api-smithi043-82244-6"}]: dispatch 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82170-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82119-7"}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82052-7"}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81739-16"}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82977-2"}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82912-2"}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81747-22"}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: osdmap e95: 8 total, 8 up, 8 in 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "overlaypool": "test-rados-api-smithi043-82244-6"}]: dispatch 2024-02-05T07:37:11.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:11.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:11.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: Running main() from gmock_main.cc 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment set-up. 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (337 ms) 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockShared (56 ms) 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-05T07:37:11.861 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1027 ms) 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1008 ms) 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.Unlock (8 ms) 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (70 ms) 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (4 ms) 2024-02-05T07:37:11.862 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock (2515 ms total) 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1131 ms) 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (3 ms) 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1066 ms) 2024-02-05T07:37:11.863 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-05T07:37:11.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1132 ms) 2024-02-05T07:37:11.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-05T07:37:11.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-02-05T07:37:11.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-05T07:37:11.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (5 ms) 2024-02-05T07:37:11.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-05T07:37:11.864 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-02-05T07:37:11.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-05T07:37:11.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (3 ms) 2024-02-05T07:37:11.865 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3350 ms total) 2024-02-05T07:37:11.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-02-05T07:37:11.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment tear-down 2024-02-05T07:37:11.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (28175 ms total) 2024-02-05T07:37:11.866 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-05T07:37:12.442 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: pgmap v98: 752 pgs: 292 unknown, 8 creating+peering, 452 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:12.442 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:12.442 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:12.442 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: pool 'PoolQuotaPP_smithi043-81716-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-05T07:37:12.442 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T07:37:12.443 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "overlaypool": "test-rados-api-smithi043-82244-6"}]': finished 2024-02-05T07:37:12.443 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:12.443 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:12.443 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:12.443 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:12.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1651010625' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:12.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:12.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: osdmap e96: 8 total, 8 up, 8 in 2024-02-05T07:37:12.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82244-6", "mode": "writeback"}]: dispatch 2024-02-05T07:37:12.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:12.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:12.445 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:12.445 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:12 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:12.518 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: pgmap v98: 752 pgs: 292 unknown, 8 creating+peering, 452 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: pool 'PoolQuotaPP_smithi043-81716-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "overlaypool": "test-rados-api-smithi043-82244-6"}]': finished 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1651010625' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:12.519 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: osdmap e96: 8 total, 8 up, 8 in 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82244-6", "mode": "writeback"}]: dispatch 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: pgmap v98: 752 pgs: 292 unknown, 8 creating+peering, 452 active+clean; 145 MiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:12.520 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: pool 'PoolQuotaPP_smithi043-81716-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "overlaypool": "test-rados-api-smithi043-82244-6"}]': finished 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82977-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/985771182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:12.521 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1651010625' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:12.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:12.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: osdmap e96: 8 total, 8 up, 8 in 2024-02-05T07:37:12.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82244-6", "mode": "writeback"}]: dispatch 2024-02-05T07:37:12.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:12.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]: dispatch 2024-02-05T07:37:12.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:12.522 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:12 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:12.944 INFO:tasks.workunit.client.0.smithi043.stdout: pool: Running main() from gmock_main.cc 2024-02-05T07:37:12.944 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-02-05T07:37:12.944 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] Global test environment set-up. 2024-02-05T07:37:12.944 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolList (10297 ms) 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (3138 ms) 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (4563 ms) 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (5685 ms) 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2165 ms) 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (3008 ms) 2024-02-05T07:37:12.945 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] 6 tests from NeoRadosPools (28856 ms total) 2024-02-05T07:37:12.946 INFO:tasks.workunit.client.0.smithi043.stdout: pool: 2024-02-05T07:37:12.946 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] Global test environment tear-down 2024-02-05T07:37:12.946 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [==========] 6 tests from 1 test suite ran. (28857 ms total) 2024-02-05T07:37:12.946 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ PASSED ] 6 tests. 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: pgmap v101: 656 pgs: 72 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 6.0 KiB/s wr, 4 op/s 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82244-6", "mode": "writeback"}]': finished 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2592498468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: osdmap e97: 8 total, 8 up, 8 in 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.489 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: osdmap e98: 8 total, 8 up, 8 in 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2666923818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2766341521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.490 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:13 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: pgmap v101: 656 pgs: 72 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 6.0 KiB/s wr, 4 op/s 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82244-6", "mode": "writeback"}]': finished 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2592498468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: osdmap e97: 8 total, 8 up, 8 in 2024-02-05T07:37:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: osdmap e98: 8 total, 8 up, 8 in 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: pgmap v101: 656 pgs: 72 creating+peering, 96 unknown, 488 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 6.0 KiB/s wr, 4 op/s 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82244-6", "mode": "writeback"}]': finished 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81795-10"}]': finished 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81692-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2592498468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: osdmap e97: 8 total, 8 up, 8 in 2024-02-05T07:37:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82977-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:13.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: osdmap e98: 8 total, 8 up, 8 in 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2666923818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2766341521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2666923818' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81996-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3788543068' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2766341521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:13.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:13 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:14.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-05T07:37:14.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-05T07:37:14.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-05T07:37:14.022 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (408 ms) 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94156299425056 err -107 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (1012 ms) 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94156299444800 err -107 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1007 ms) 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 352187318273 cookie 94156299444800 2024-02-05T07:37:14.023 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (11 ms) 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 356482285570 cookie 94156299481264 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (82 ms) 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 356482285568 cookie 94156299467552 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (6 ms) 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 356482285570 cookie 94156299465376 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 356482285570 cookie 94156299491776 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (8 ms) 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-05T07:37:14.024 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 356482285569 cookie 94156299491776 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 360777252866 cookie 94156299491776 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3006 ms) 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94156299491776 err -107 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24728 notify_id 382252089348 cookie 94156299491776 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5008 ms) 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94156299491776 err -107 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1006 ms) 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (11554 ms total) 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-02-05T07:37:14.025 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (935 ms) 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (935 ms total) 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (30203 ms total) 2024-02-05T07:37:14.026 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-05T07:37:14.382 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-05T07:37:14.382 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-05T07:37:14.382 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-05T07:37:14.382 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-05T07:37:14.382 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5413 ms) 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (25 ms) 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20016 ms) 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 0 of 20 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: failed to deregister: 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: { 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "epoch": 20, 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "modified": "2024-02-05T07:37:09.671674+0000", 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "services": { 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "laundry": { 2024-02-05T07:37:14.383 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "close-test-pid82342": { 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 20, 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-02-05T07:37:09.365224+0000", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 25364, 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/3015067924", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-05T07:37:14.384 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32494692", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:14.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid82311": { 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 19, 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-02-05T07:36:54.412229+0000", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24955, 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/1725810983", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-02-05T07:37:14.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32494692", 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:14.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "rgw": { 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "24358": { 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 12, 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-02-05T07:34:05.132267+0000", 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24358, 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/2658810356", 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:37:14.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_hostname": "smithi043", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "8", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "id": "foo.a", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:37:14.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32494692", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "num_handles": "1", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid": "7", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_id": "", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_name": "", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_id": "f1334c69-2d4a-42d5-af58-ea4f5c71c43a", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_name": "default", 2024-02-05T07:37:14.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_id": "b9c65ceb-129c-45da-adcf-32cc35da96f7", 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_name": "defau api_service: [==========] Running 4 tests from 1 test suite. 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment set-up. 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5071 ms) 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterLate (27 ms) 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-02-05T07:37:14.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_OK 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 239 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 132 active+clean 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_WARN 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 31 pool(s) do not have an application enabled 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-02-05T07:37:14.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 41 pools, 1252 pgs 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 198 objects, 582 KiB 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 240 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 31.869% pgs unknown 2024-02-05T07:37:14.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 55.990% pgs not active 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 701 creating+peering 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 399 unknown 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 152 active+clean 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 852 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.StatusFormat (5271 ms) 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.Status (20146 ms) 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService (30517 ms total) 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-02-05T07:37:14.406 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment tear-down 2024-02-05T07:37:14.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [==========] 4 tests from 1 test suite ran. (30517 ms total) 2024-02-05T07:37:14.407 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ PASSED ] 4 tests. 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2666923818' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81996-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: osdmap e99: 8 total, 8 up, 8 in 2024-02-05T07:37:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2666923818' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81996-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: osdmap e99: 8 total, 8 up, 8 in 2024-02-05T07:37:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]: dispatch 2024-02-05T07:37:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2666923818' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81996-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82170-12"}]': finished 2024-02-05T07:37:14.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:14.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: osdmap e99: 8 total, 8 up, 8 in 2024-02-05T07:37:14.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1287 ms) 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-05T07:37:15.190 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (48 ms) 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1335 ms total) 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (120 ms) 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4229 ms) 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (6 ms) 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (5 ms) 2024-02-05T07:37:15.191 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187675072 notify_id 416611827713 notifier_gid 15276 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (6 ms) 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187672160 notify_id 416611827714 notifier_gid 15276 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187675072 notify_id 416611827715 notifier_gid 15276 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (4 ms) 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187675072 notify_id 416611827716 notifier_gid 15276 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (5 ms) 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187675072 notify_id 416611827716 notifier_gid 15276 2024-02-05T07:37:15.192 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187677984 notify_id 416611827717 notifier_gid 15276 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (5 ms) 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187677984 notify_id 416611827717 notifier_gid 15276 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-02-05T07:37:15.193 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: pgmap v104: 624 pgs: 200 unknown, 424 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1986521541' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: osdmap e100: 8 total, 8 up, 8 in 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3832494046' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3387411655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: pgmap v104: 624 pgs: 200 unknown, 424 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1986521541' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:16.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: osdmap e100: 8 total, 8 up, 8 in 2024-02-05T07:37:16.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3832494046' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3387411655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: pgmap v104: 624 pgs: 200 unknown, 424 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 1 op/s 2024-02-05T07:37:16.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:16.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1986521541' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:16.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:16.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:16.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:16.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: osdmap e100: 8 total, 8 up, 8 in 2024-02-05T07:37:16.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3832494046' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3387411655' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:16.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: pgmap v107: 688 pgs: 264 unknown, 424 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:17.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: osdmap e101: 8 total, 8 up, 8 in 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4181640116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:37:17.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: pgmap v107: 688 pgs: 264 unknown, 424 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: osdmap e101: 8 total, 8 up, 8 in 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4181640116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:17.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:16 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:17.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: pgmap v107: 688 pgs: 264 unknown, 424 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81692-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: osdmap e101: 8 total, 8 up, 8 in 2024-02-05T07:37:17.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4181640116' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:17.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:17.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:17.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:16 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: osdmap e102: 8 total, 8 up, 8 in 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:18.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:18.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: osdmap e102: 8 total, 8 up, 8 in 2024-02-05T07:37:18.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:18.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:18.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:18.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:18.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:18.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: osdmap e102: 8 total, 8 up, 8 in 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:18.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:19.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_wat ec_list: Running main() from gmock_main.cc 2024-02-05T07:37:19.007 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-02-05T07:37:19.007 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] Global test environment set-up. 2024-02-05T07:37:19.007 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (15847 ms) 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (9969 ms) 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (9104 ms) 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] 3 tests from NeoradosECList (34920 ms total) 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] Global test environment tear-down 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (34920 ms total) 2024-02-05T07:37:19.008 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ PASSED ] 3 tests. 2024-02-05T07:37:19.312 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:37:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]': finished 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: osdmap e103: 8 total, 8 up, 8 in 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2824810058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1707034782' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82081-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]': finished 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: osdmap e103: 8 total, 8 up, 8 in 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2824810058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1707034782' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82081-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-7"}]': finished 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: osdmap e103: 8 total, 8 up, 8 in 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2824810058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1707034782' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82081-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3894317536' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]: dispatch 2024-02-05T07:37:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:19.616 INFO:tasks.workunit.client.0.smithi043.stdout:lt" 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 1 of 20 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: failed to deregister: 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: { 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "epoch": 22, 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "modified": "2024-02-05T07:37:15.262896+0000", 2024-02-05T07:37:19.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "services": { 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "laundry": { 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "close-test-pid82342": { 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 22, 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-02-05T07:37:14.599591+0000", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 15396, 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/3107811520", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-05T07:37:19.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 9", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "9", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "foo": "bar", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32494692", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "this": "that" 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:19.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "rgw": { 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "daemons": { 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "summary": "", 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "24358": { 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_epoch": 12, 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "start_stamp": "2024-02-05T07:34:05.132267+0000", 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "gid": 24358, 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "addr": "172.21.15.43:0/2658810356", 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "metadata": { 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "arch": "x86_64", 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_release": "squid", 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version": "ceph version 19.0.0-1241-g164d3be4 (164d3be4a527be775f4d3c343ea591122f1b8921) squid (dev)", 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "ceph_version_short": "19.0.0-1241-g164d3be4", 2024-02-05T07:37:19.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_hostname": "smithi043", 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: pgmap v110: 648 pgs: 160 unknown, 488 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 3 op/s 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1707034782' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82081-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: osdmap e104: 8 total, 8 up, 8 in 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1234270882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/945804621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2527413679' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: pgmap v110: 648 pgs: 160 unknown, 488 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 3 op/s 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1707034782' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82081-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: osdmap e104: 8 total, 8 up, 8 in 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1234270882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:20.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/945804621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:20.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2527413679' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:20.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: pgmap v110: 648 pgs: 160 unknown, 488 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 3 op/s 2024-02-05T07:37:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1707034782' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82081-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81692-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82977-3"}]': finished 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: osdmap e104: 8 total, 8 up, 8 in 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1234270882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/945804621' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2527413679' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:20.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:21.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:37:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:37:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: pgmap v113: 680 pgs: 256 unknown, 424 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 3 op/s 2024-02-05T07:37:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1234270882' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/945804621' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: osdmap e105: 8 total, 8 up, 8 in 2024-02-05T07:37:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:21.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2868964495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:21.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:21.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:21.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:21.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:21 smithi050 ceph-mon[33492]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: pgmap v113: 680 pgs: 256 unknown, 424 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 3 op/s 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1234270882' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/945804621' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: osdmap e105: 8 total, 8 up, 8 in 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2868964495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[29879]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: pgmap v113: 680 pgs: 256 unknown, 424 active+clean; 145 MiB data, 760 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 3 op/s 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1234270882' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/945804621' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82365-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: osdmap e105: 8 total, 8 up, 8 in 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2868964495' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:21.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:21.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]: dispatch 2024-02-05T07:37:21.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:21.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:21 smithi043 ceph-mon[41444]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:22.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b16616 write_operations: Running main() from gmock_main.cc 2024-02-05T07:37:22.041 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-02-05T07:37:22.041 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] Global test environment set-up. 2024-02-05T07:37:22.041 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-02-05T07:37:22.041 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-02-05T07:37:22.041 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (13412 ms) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (4826 ms) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (5397 ms) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3314 ms) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3948 ms) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (2892 ms) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (4122 ms) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (37911 ms total) 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: 2024-02-05T07:37:22.042 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] Global test environment tear-down 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (37911 ms total) 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ PASSED ] 7 tests. 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-05T07:37:22.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (13610 ms) 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (7128 ms) 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3132 ms) 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-05T07:37:22.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (4366 ms) 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (2901 ms) 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (2884 ms) 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4121 ms) 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (38143 ms total) 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (38143 ms total) 2024-02-05T07:37:22.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]': finished 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[29879]: osdmap e106: 8 total, 8 up, 8 in 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1560884744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]': finished 2024-02-05T07:37:22.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[41444]: osdmap e106: 8 total, 8 up, 8 in 2024-02-05T07:37:22.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1560884744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:22.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:22.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:22 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-7"}]': finished 2024-02-05T07:37:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:22 smithi050 ceph-mon[33492]: osdmap e106: 8 total, 8 up, 8 in 2024-02-05T07:37:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1560884744' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[29879]: pgmap v116: 680 pgs: 64 creating+peering, 96 unknown, 520 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-02-05T07:37:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[29879]: osdmap e107: 8 total, 8 up, 8 in 2024-02-05T07:37:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1663337775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:23.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[41444]: pgmap v116: 680 pgs: 64 creating+peering, 96 unknown, 520 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-02-05T07:37:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[41444]: osdmap e107: 8 total, 8 up, 8 in 2024-02-05T07:37:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1663337775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:23.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:23.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:23 smithi050 ceph-mon[33492]: pgmap v116: 680 pgs: 64 creating+peering, 96 unknown, 520 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-02-05T07:37:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81692-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:23 smithi050 ceph-mon[33492]: osdmap e107: 8 total, 8 up, 8 in 2024-02-05T07:37:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1663337775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[29879]: osdmap e108: 8 total, 8 up, 8 in 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[41444]: osdmap e108: 8 total, 8 up, 8 in 2024-02-05T07:37:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81716-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:24 smithi050 ceph-mon[33492]: osdmap e108: 8 total, 8 up, 8 in 2024-02-05T07:37:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout:5464b0", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro": "centos", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "distro_version": "8", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "hostname": "smithi043", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "id": "foo.a", 2024-02-05T07:37:24.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_description": "#1 SMP PREEMPT_DYNAMIC Wed Jan 31 22:52:48 UTC 2024", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "kernel_version": "5.14.0-416.el9.x86_64", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_swap_kb": "0", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "mem_total_kb": "32494692", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "num_handles": "1", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "os": "Linux", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "pid": "7", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_id": "", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "realm_name": "", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_id": "f1334c69-2d4a-42d5-af58-ea4f5c71c43a", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zone_name": "default", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_id": "b9c65ceb-129c-45da-adcf-32cc35da96f7", 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: }, 2024-02-05T07:37:24.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: "task_status": {} 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: } 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 2 of 20 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (15323 ms) 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (40777 ms total) 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (40777 ms total) 2024-02-05T07:37:24.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-05T07:37:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: pgmap v119: 584 pgs: 64 creating+peering, 96 unknown, 424 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:37:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: osdmap e109: 8 total, 8 up, 8 in 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/129388543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4"}]: dispatch 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3679251228' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: pgmap v119: 584 pgs: 64 creating+peering, 96 unknown, 424 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: osdmap e109: 8 total, 8 up, 8 in 2024-02-05T07:37:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/129388543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4"}]: dispatch 2024-02-05T07:37:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3679251228' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: pgmap v119: 584 pgs: 64 creating+peering, 96 unknown, 424 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-05T07:37:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: osdmap e109: 8 total, 8 up, 8 in 2024-02-05T07:37:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/129388543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4"}]: dispatch 2024-02-05T07:37:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3679251228' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:26.505 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:26.505 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.505 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4"}]': finished 2024-02-05T07:37:26.505 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:26.505 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: osdmap e110: 8 total, 8 up, 8 in 2024-02-05T07:37:26.505 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6"}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/313315002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3975811806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6"}]': finished 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]': finished 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.506 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: osdmap e111: 8 total, 8 up, 8 in 2024-02-05T07:37:26.507 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]: dispatch 2024-02-05T07:37:26.507 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82244-6", "pool2": "test-rados-api-smithi043-82244-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.507 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]: dispatch 2024-02-05T07:37:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4"}]': finished 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: osdmap e110: 8 total, 8 up, 8 in 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6"}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/313315002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3975811806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6"}]': finished 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]': finished 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: osdmap e111: 8 total, 8 up, 8 in 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]: dispatch 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82244-6", "pool2": "test-rados-api-smithi043-82244-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]: dispatch 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81692-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4"}]': finished 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: osdmap e110: 8 total, 8 up, 8 in 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6"}]: dispatch 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/313315002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3975811806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82244-4", "tierpool": "test-rados-api-smithi043-82244-6"}]': finished 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-9"}]': finished 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81716-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: osdmap e111: 8 total, 8 up, 8 in 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]: dispatch 2024-02-05T07:37:26.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82244-6", "pool2": "test-rados-api-smithi043-82244-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T07:37:26.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]: dispatch 2024-02-05T07:37:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: pgmap v122: 680 pgs: 32 creating+peering, 224 unknown, 424 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:37:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:37:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:37:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:37:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82244-6", "pool2": "test-rados-api-smithi043-82244-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]': finished 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3910679713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: osdmap e112: 8 total, 8 up, 8 in 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:27.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: pgmap v122: 680 pgs: 32 creating+peering, 224 unknown, 424 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:37:27.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:37:27.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82244-6", "pool2": "test-rados-api-smithi043-82244-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]': finished 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3910679713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: osdmap e112: 8 total, 8 up, 8 in 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:27.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:27 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: pgmap v122: 680 pgs: 32 creating+peering, 224 unknown, 424 active+clean; 145 MiB data, 775 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.0"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.1"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.2"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.3"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.4"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:37:27.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.5"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.6"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.7"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.8"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "19.9"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/338928195' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82244-6", "pool2": "test-rados-api-smithi043-82244-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-9", "mode": "writeback"}]': finished 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3910679713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:27.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: osdmap e112: 8 total, 8 up, 8 in 2024-02-05T07:37:27.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:27.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:27.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:27.825 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:27 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:28.048 INFO:tasks.workunit.client.0.smithi043.stdout:ch_notify_pp: flushed 2024-02-05T07:37:28.048 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3005 ms) 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187677984 notify_id 429496729606 notifier_gid 15276 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushed 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3005 ms) 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187677984 notify_id 442381631494 notifier_gid 15276 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-02-05T07:37:28.049 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3006 ms) 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 94023187677984 notify_id 455266533383 notifier_gid 15276 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3006 ms) 2024-02-05T07:37:28.050 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16413 ms total) 2024-02-05T07:37:28.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-02-05T07:37:28.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-05T07:37:28.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (44200 ms total) 2024-02-05T07:37:28.051 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.3 deep-scrub starts 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.3 deep-scrub ok 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: pgmap v125: 644 pgs: 128 unknown, 64 creating+peering, 452 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.7 deep-scrub starts 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.7 deep-scrub ok 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.6 deep-scrub starts 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.6 deep-scrub ok 2024-02-05T07:37:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.2 deep-scrub starts 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.2 deep-scrub ok 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.1 deep-scrub starts 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: 19.1 deep-scrub ok 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: osdmap e113: 8 total, 8 up, 8 in 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.3 deep-scrub starts 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.3 deep-scrub ok 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: pgmap v125: 644 pgs: 128 unknown, 64 creating+peering, 452 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.7 deep-scrub starts 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.7 deep-scrub ok 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.6 deep-scrub starts 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.6 deep-scrub ok 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.2 deep-scrub starts 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.2 deep-scrub ok 2024-02-05T07:37:28.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.1 deep-scrub starts 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: 19.1 deep-scrub ok 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: osdmap e113: 8 total, 8 up, 8 in 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:28.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.3 deep-scrub starts 2024-02-05T07:37:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.3 deep-scrub ok 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: pgmap v125: 644 pgs: 128 unknown, 64 creating+peering, 452 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.7 deep-scrub starts 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.7 deep-scrub ok 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.6 deep-scrub starts 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.6 deep-scrub ok 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.2 deep-scrub starts 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.2 deep-scrub ok 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.1 deep-scrub starts 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: 19.1 deep-scrub ok 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81692-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: osdmap e113: 8 total, 8 up, 8 in 2024-02-05T07:37:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:28.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]: dispatch 2024-02-05T07:37:28.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:28.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:28.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:37:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:37:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:37:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.0 deep-scrub starts 2024-02-05T07:37:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.0 deep-scrub ok 2024-02-05T07:37:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.5 deep-scrub starts 2024-02-05T07:37:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.5 deep-scrub ok 2024-02-05T07:37:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.9 deep-scrub starts 2024-02-05T07:37:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.9 deep-scrub ok 2024-02-05T07:37:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.4 deep-scrub starts 2024-02-05T07:37:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: 19.4 deep-scrub ok 2024-02-05T07:37:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:30.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:30.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]': finished 2024-02-05T07:37:30.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:30.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: osdmap e114: 8 total, 8 up, 8 in 2024-02-05T07:37:30.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:30.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/640133675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81716-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:30.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:30.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.0 deep-scrub starts 2024-02-05T07:37:30.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.0 deep-scrub ok 2024-02-05T07:37:30.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.5 deep-scrub starts 2024-02-05T07:37:30.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.5 deep-scrub ok 2024-02-05T07:37:30.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.9 deep-scrub starts 2024-02-05T07:37:30.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.9 deep-scrub ok 2024-02-05T07:37:30.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.4 deep-scrub starts 2024-02-05T07:37:30.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: 19.4 deep-scrub ok 2024-02-05T07:37:30.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:30.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:30.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]': finished 2024-02-05T07:37:30.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:30.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: osdmap e114: 8 total, 8 up, 8 in 2024-02-05T07:37:30.568 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:30.568 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/640133675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81716-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:30.568 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:30.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.0 deep-scrub starts 2024-02-05T07:37:30.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.0 deep-scrub ok 2024-02-05T07:37:30.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.5 deep-scrub starts 2024-02-05T07:37:30.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.5 deep-scrub ok 2024-02-05T07:37:30.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.9 deep-scrub starts 2024-02-05T07:37:30.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.9 deep-scrub ok 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.4 deep-scrub starts 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: 19.4 deep-scrub ok 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-9"}]': finished 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: osdmap e114: 8 total, 8 up, 8 in 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/640133675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81716-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:30.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:31.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:37:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:37:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: pgmap v128: 484 pgs: 96 unknown, 388 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: 19.8 deep-scrub starts 2024-02-05T07:37:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: 19.8 deep-scrub ok 2024-02-05T07:37:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/640133675' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81716-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: osdmap e115: 8 total, 8 up, 8 in 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2265518358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81692-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2265518358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81692-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[29879]: osdmap e116: 8 total, 8 up, 8 in 2024-02-05T07:37:31.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: pgmap v128: 484 pgs: 96 unknown, 388 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: 19.8 deep-scrub starts 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: 19.8 deep-scrub ok 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/640133675' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81716-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: osdmap e115: 8 total, 8 up, 8 in 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2265518358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81692-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2265518358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81692-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:31 smithi043 ceph-mon[41444]: osdmap e116: 8 total, 8 up, 8 in 2024-02-05T07:37:31.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: pgmap v128: 484 pgs: 96 unknown, 388 active+clean; 145 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:31.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: 19.8 deep-scrub starts 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: 19.8 deep-scrub ok 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/640133675' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81716-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: osdmap e115: 8 total, 8 up, 8 in 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2265518358' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81692-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81996-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82057-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2265518358' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81692-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:31 smithi050 ceph-mon[33492]: osdmap e116: 8 total, 8 up, 8 in 2024-02-05T07:37:33.029 INFO:tasks.workunit.client.0.smithi043.stdout: misc: Running main() from gmock_main.cc 2024-02-05T07:37:33.029 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-02-05T07:37:33.029 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] Global test environment set-up. 2024-02-05T07:37:33.029 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-02-05T07:37:33.029 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-02-05T07:37:33.029 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Version (9958 ms) 2024-02-05T07:37:33.029 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (2727 ms) 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongName (3694 ms) 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4168 ms) 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (5302 ms) 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (3007 ms) 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Exec (2968 ms) 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (4102 ms) 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-02-05T07:37:33.030 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3022 ms) 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.BigObject (2916 ms) 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (3095 ms) 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3976 ms) 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] 12 tests from NeoRadosMisc (48935 ms total) 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] Global test environment tear-down 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [==========] 12 tests from 1 test suite ran. (48936 ms total) 2024-02-05T07:37:33.031 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ PASSED ] 12 tests. 2024-02-05T07:37:33.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:33.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:33.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: osdmap e117: 8 total, 8 up, 8 in 2024-02-05T07:37:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache", "force_nonempty":""}]: dispatch 2024-02-05T07:37:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/922598433' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.069 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:33.069 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:33.069 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: osdmap e117: 8 total, 8 up, 8 in 2024-02-05T07:37:33.069 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache", "force_nonempty":""}]: dispatch 2024-02-05T07:37:33.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/922598433' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.070 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: osdmap e117: 8 total, 8 up, 8 in 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache", "force_nonempty":""}]: dispatch 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/922598433' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: pgmap v131: 524 pgs: 128 creating+peering, 40 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 3 op/s 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache", "force_nonempty":""}]': finished 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: osdmap e118: 8 total, 8 up, 8 in 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82057-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2647269922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: pgmap v131: 524 pgs: 128 creating+peering, 40 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 3 op/s 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache", "force_nonempty":""}]': finished 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: osdmap e118: 8 total, 8 up, 8 in 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82057-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2647269922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: pgmap v131: 524 pgs: 128 creating+peering, 40 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 3 op/s 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache", "force_nonempty":""}]': finished 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81716-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: osdmap e118: 8 total, 8 up, 8 in 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82057-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2647269922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: pgmap v134: 588 pgs: 32 creating+peering, 200 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82057-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: osdmap e119: 8 total, 8 up, 8 in 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache"}]: dispatch 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:35.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: pgmap v134: 588 pgs: 32 creating+peering, 200 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82057-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: osdmap e119: 8 total, 8 up, 8 in 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache"}]: dispatch 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:35.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:35.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: pgmap v134: 588 pgs: 32 creating+peering, 200 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 18 MiB/s wr, 6 op/s 2024-02-05T07:37:35.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:35.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82057-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:35.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81692-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:35.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: osdmap e119: 8 total, 8 up, 8 in 2024-02-05T07:37:35.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache"}]: dispatch 2024-02-05T07:37:35.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:35.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:36.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:36.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache"}]': finished 2024-02-05T07:37:36.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:36.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[29879]: osdmap e120: 8 total, 8 up, 8 in 2024-02-05T07:37:36.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:36.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:36.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2385524023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:36.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache"}]': finished 2024-02-05T07:37:36.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:36.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[41444]: osdmap e120: 8 total, 8 up, 8 in 2024-02-05T07:37:36.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:36.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:36.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2385524023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:36.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4220483301' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-82057-10", "tierpool":"test-rados-api-smithi043-82057-10-cache"}]': finished 2024-02-05T07:37:36.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:36.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:35 smithi050 ceph-mon[33492]: osdmap e120: 8 total, 8 up, 8 in 2024-02-05T07:37:36.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:36.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:36.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2385524023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: pgmap v137: 556 pgs: 32 creating+peering, 168 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]': finished 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2385524023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: osdmap e121: 8 total, 8 up, 8 in 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]: dispatch 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]: dispatch 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3861588109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: pgmap v137: 556 pgs: 32 creating+peering, 168 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]': finished 2024-02-05T07:37:36.965 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2385524023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:36.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: osdmap e121: 8 total, 8 up, 8 in 2024-02-05T07:37:36.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]: dispatch 2024-02-05T07:37:36.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]: dispatch 2024-02-05T07:37:36.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3861588109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.966 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: pgmap v137: 556 pgs: 32 creating+peering, 168 unknown, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-11"}]': finished 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2385524023' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: osdmap e121: 8 total, 8 up, 8 in 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]: dispatch 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]: dispatch 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3861588109' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:36.984 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:37.072 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: Running main() from gmock_main.cc 2024-02-05T07:37:37.072 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-05T07:37:37.072 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment set-up. 2024-02-05T07:37:37.072 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-05T07:37:37.072 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-05T07:37:37.072 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolList (10610 ms) 2024-02-05T07:37:37.072 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (6898 ms) 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4412 ms) 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (4255 ms) 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (4072 ms) 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (4991 ms) 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5026 ms) 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4995 ms) 2024-02-05T07:37:37.073 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-05T07:37:37.074 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8032 ms) 2024-02-05T07:37:37.074 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools (53291 ms total) 2024-02-05T07:37:37.074 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: 2024-02-05T07:37:37.074 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment tear-down 2024-02-05T07:37:37.074 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (53291 ms total) 2024-02-05T07:37:37.074 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]': finished 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[29879]: osdmap e122: 8 total, 8 up, 8 in 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]': finished 2024-02-05T07:37:38.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:38.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:37 smithi043 ceph-mon[41444]: osdmap e122: 8 total, 8 up, 8 in 2024-02-05T07:37:38.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:38.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:37 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:38.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-11", "mode": "writeback"}]': finished 2024-02-05T07:37:38.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81692-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:38.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:37 smithi050 ceph-mon[33492]: osdmap e122: 8 total, 8 up, 8 in 2024-02-05T07:37:39.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: pgmap v140: 492 pgs: 96 unknown, 396 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:37:39.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:39.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:39.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: osdmap e123: 8 total, 8 up, 8 in 2024-02-05T07:37:39.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2974941596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:39.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:39.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:39.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:39.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: pgmap v140: 492 pgs: 96 unknown, 396 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:37:39.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:39.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:39.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: osdmap e123: 8 total, 8 up, 8 in 2024-02-05T07:37:39.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2974941596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:39.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:39.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:39.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:39.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: pgmap v140: 492 pgs: 96 unknown, 396 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:37:39.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:39.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:39.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: osdmap e123: 8 total, 8 up, 8 in 2024-02-05T07:37:39.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2974941596' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:39.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:39.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:39.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:37:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:37:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:37:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[29879]: osdmap e124: 8 total, 8 up, 8 in 2024-02-05T07:37:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3353644678' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81692-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[41444]: osdmap e124: 8 total, 8 up, 8 in 2024-02-05T07:37:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3353644678' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81692-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:40.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:40.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81716-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:37:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:39 smithi050 ceph-mon[33492]: osdmap e124: 8 total, 8 up, 8 in 2024-02-05T07:37:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]: dispatch 2024-02-05T07:37:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3353644678' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81692-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[41444]: pgmap v143: 524 pgs: 128 unknown, 396 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]': finished 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3353644678' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81692-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[41444]: osdmap e125: 8 total, 8 up, 8 in 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[29879]: pgmap v143: 524 pgs: 128 unknown, 396 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:37:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]': finished 2024-02-05T07:37:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3353644678' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81692-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:41.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:40 smithi043 ceph-mon[29879]: osdmap e125: 8 total, 8 up, 8 in 2024-02-05T07:37:41.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:37:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:37:41.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:40 smithi050 ceph-mon[33492]: pgmap v143: 524 pgs: 128 unknown, 396 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-02-05T07:37:41.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:40 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:37:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-11"}]': finished 2024-02-05T07:37:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3353644678' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81692-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:41.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:40 smithi050 ceph-mon[33492]: osdmap e125: 8 total, 8 up, 8 in 2024-02-05T07:37:41.961 INFO:tasks.workunit.client.0.smithi043.stdout: io: Running main() from gmock_main.cc 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] Global test environment set-up. 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] 14 tests from NeoRadosIo 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Limits (13440 ms) 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (5588 ms) 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.ReadOp (6086 ms) 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.SparseRead (2960 ms) 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-02-05T07:37:41.962 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.RoundTrip (2892 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (2891 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (3110 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3020 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3244 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Trunc (3081 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Remove (2656 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3012 ms) 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-02-05T07:37:41.963 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.RmXattr (3006 ms) 2024-02-05T07:37:41.964 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-02-05T07:37:41.964 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2913 ms) 2024-02-05T07:37:41.964 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] 14 tests from NeoRadosIo (57900 ms total) 2024-02-05T07:37:41.964 INFO:tasks.workunit.client.0.smithi043.stdout: io: 2024-02-05T07:37:41.964 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] Global test environment tear-down 2024-02-05T07:37:41.964 INFO:tasks.workunit.client.0.smithi043.stdout: io: [==========] 14 tests from 1 test suite ran. (57900 ms total) 2024-02-05T07:37:41.964 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ PASSED ] 14 tests. 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[29879]: osdmap e126: 8 total, 8 up, 8 in 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2038740846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81716-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:42.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[41444]: osdmap e126: 8 total, 8 up, 8 in 2024-02-05T07:37:42.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2038740846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81716-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:42.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:42.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:41 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:41 smithi050 ceph-mon[33492]: osdmap e126: 8 total, 8 up, 8 in 2024-02-05T07:37:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2038740846' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81716-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: pgmap v146: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2038740846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81716-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: osdmap e127: 8 total, 8 up, 8 in 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:43.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/118888260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81692-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: pgmap v146: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2038740846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81716-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: osdmap e127: 8 total, 8 up, 8 in 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:43.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/118888260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81692-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:43.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:42 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: pgmap v146: 452 pgs: 32 creating+peering, 32 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2038740846' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81716-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: osdmap e127: 8 total, 8 up, 8 in 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1054979994' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]: dispatch 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/118888260' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81692-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:42 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:43.970 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: Running main() from gmock_main.cc 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] Global test environment set-up. 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (13393 ms) 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (7134 ms) 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (5304 ms) 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3000 ms) 2024-02-05T07:37:43.971 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (2973 ms) 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (4103 ms) 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3041 ms) 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (2895 ms) 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3100 ms) 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2967 ms) 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3024 ms) 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-02-05T07:37:43.972 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3001 ms) 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2914 ms) 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3009 ms) 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (59858 ms total) 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] Global test environment tear-down 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (59858 ms total) 2024-02-05T07:37:43.973 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ PASSED ] 14 tests. 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/118888260' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81692-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: osdmap e128: 8 total, 8 up, 8 in 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/118888260' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81692-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: osdmap e128: 8 total, 8 up, 8 in 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:44.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81996-10"}]': finished 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/118888260' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81692-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: osdmap e128: 8 total, 8 up, 8 in 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: pgmap v149: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: osdmap e129: 8 total, 8 up, 8 in 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/251524005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81716-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:45.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: pgmap v149: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: osdmap e129: 8 total, 8 up, 8 in 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/251524005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81716-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:45.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: pgmap v149: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-02-05T07:37:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:37:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: osdmap e129: 8 total, 8 up, 8 in 2024-02-05T07:37:45.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/251524005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81716-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:45.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/251524005' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81716-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[29879]: osdmap e130: 8 total, 8 up, 8 in 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/506041815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/251524005' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81716-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[41444]: osdmap e130: 8 total, 8 up, 8 in 2024-02-05T07:37:46.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/506041815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:46.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:46.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:46.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/251524005' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81716-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:46.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:45 smithi050 ceph-mon[33492]: osdmap e130: 8 total, 8 up, 8 in 2024-02-05T07:37:46.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/506041815' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:46.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment set-up. 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-02-05T07:36:43.812+0000 7fa80c9b7780 -1 failed for service _ceph-mon._tcp 2024-02-05T07:37:46.385 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-02-05T07:36:43.812+0000 7fa80c9b7780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (59 ms) 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2009 ms) 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2068 ms total) 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567be5e720 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19b680 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19c000 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19bb60 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19cb20 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19c950 2024-02-05T07:37:46.386 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19d030 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19db70 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19da30 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19dc80 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e140 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19dde0 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19def0 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e250 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e6e0 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e7f0 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19d670 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19d7e0 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19d8f0 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e900 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19ea50 2024-02-05T07:37:46.387 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f000 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19eb60 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19ec70 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19ed80 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19eeb0 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f5b0 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f110 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f240 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f350 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f4a0 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19fb60 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f6c0 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f7d0 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19f900 2024-02-05T07:37:46.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19fa50 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0150 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19fcb0 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19fdc0 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19fed0 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0020 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0720 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0280 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0390 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a04a0 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a05b0 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0d10 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0870 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a09c0 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0ad0 2024-02-05T07:37:46.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0c00 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[41444]: pgmap v152: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[41444]: osdmap e131: 8 total, 8 up, 8 in 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[29879]: pgmap v152: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:47.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:46 smithi043 ceph-mon[29879]: osdmap e131: 8 total, 8 up, 8 in 2024-02-05T07:37:47.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:46 smithi050 ceph-mon[33492]: pgmap v152: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:47.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:47.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81996-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:47.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81692-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:47.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:46 smithi050 ceph-mon[33492]: osdmap e131: 8 total, 8 up, 8 in 2024-02-05T07:37:48.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:48.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[29879]: osdmap e132: 8 total, 8 up, 8 in 2024-02-05T07:37:48.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1634333690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:48.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:48.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:48.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:48.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:48.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[41444]: osdmap e132: 8 total, 8 up, 8 in 2024-02-05T07:37:48.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1634333690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:48.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:48.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:48.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:48.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:48.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:47 smithi050 ceph-mon[33492]: osdmap e132: 8 total, 8 up, 8 in 2024-02-05T07:37:48.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1634333690' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:48.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:48.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:48.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: pgmap v155: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: osdmap e133: 8 total, 8 up, 8 in 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:37:49.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2138879452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81692-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: pgmap v155: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: osdmap e133: 8 total, 8 up, 8 in 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:37:49.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2138879452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81692-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:49.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: pgmap v155: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 73 op/s 2024-02-05T07:37:49.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81716-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:37:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: osdmap e133: 8 total, 8 up, 8 in 2024-02-05T07:37:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:37:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:37:49.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2138879452' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81692-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:37:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]': finished 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2138879452' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81692-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]: dispatch 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[29879]: osdmap e134: 8 total, 8 up, 8 in 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]: dispatch 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:50.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]': finished 2024-02-05T07:37:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2138879452' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81692-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]: dispatch 2024-02-05T07:37:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[41444]: osdmap e134: 8 total, 8 up, 8 in 2024-02-05T07:37:50.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]: dispatch 2024-02-05T07:37:50.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:50.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-13"}]': finished 2024-02-05T07:37:50.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2138879452' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81692-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:50.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]: dispatch 2024-02-05T07:37:50.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:49 smithi050 ceph-mon[33492]: osdmap e134: 8 total, 8 up, 8 in 2024-02-05T07:37:50.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]: dispatch 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: pgmap v158: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]': finished 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: osdmap e135: 8 total, 8 up, 8 in 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3322660159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: 172.8 scrub starts 2024-02-05T07:37:51.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[29879]: 172.8 scrub ok 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: pgmap v158: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]': finished 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: osdmap e135: 8 total, 8 up, 8 in 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3322660159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-05T07:37:51.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-05T07:37:51.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-05T07:37:51.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-05T07:37:51.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-05T07:37:51.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-05T07:37:51.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: 172.8 scrub starts 2024-02-05T07:37:51.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:50 smithi043 ceph-mon[41444]: 172.8 scrub ok 2024-02-05T07:37:51.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:37:51.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: pgmap v158: 460 pgs: 72 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 75 KiB/s wr, 76 op/s 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-13", "mode": "writeback"}]': finished 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: osdmap e135: 8 total, 8 up, 8 in 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3322660159' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-05T07:37:51.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-05T07:37:51.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-05T07:37:51.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-05T07:37:51.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-05T07:37:51.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-05T07:37:51.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: 172.8 scrub starts 2024-02-05T07:37:51.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:50 smithi050 ceph-mon[33492]: 172.8 scrub ok 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-05T07:37:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.9 scrub starts 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.9 scrub ok 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.2 scrub starts 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.2 scrub ok 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.4 scrub starts 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.4 scrub ok 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: osdmap e136: 8 total, 8 up, 8 in 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3037560740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81692-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:52.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.5 scrub starts 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[29879]: 172.5 scrub ok 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-05T07:37:52.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.9 scrub starts 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.9 scrub ok 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.2 scrub starts 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.2 scrub ok 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.4 scrub starts 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.4 scrub ok 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: osdmap e136: 8 total, 8 up, 8 in 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3037560740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81692-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.5 scrub starts 2024-02-05T07:37:52.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:51 smithi043 ceph-mon[41444]: 172.5 scrub ok 2024-02-05T07:37:52.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.0"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.1"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.2"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.3"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.4"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.5"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.6"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.7"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.8"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "172.9"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.9 scrub starts 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.9 scrub ok 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.2 scrub starts 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.2 scrub ok 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:37:52.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.4 scrub starts 2024-02-05T07:37:52.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.4 scrub ok 2024-02-05T07:37:52.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:52.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81716-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:52.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: osdmap e136: 8 total, 8 up, 8 in 2024-02-05T07:37:52.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3037560740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81692-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:52.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.5 scrub starts 2024-02-05T07:37:52.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:51 smithi050 ceph-mon[33492]: 172.5 scrub ok 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.7 scrub starts 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.7 scrub ok 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: pgmap v161: 460 pgs: 32 creating+peering, 32 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.0 scrub starts 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.0 scrub ok 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.3 scrub starts 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.3 scrub ok 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3037560740' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81692-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: osdmap e137: 8 total, 8 up, 8 in 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.1 scrub starts 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[29879]: 172.1 scrub ok 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.7 scrub starts 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.7 scrub ok 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: pgmap v161: 460 pgs: 32 creating+peering, 32 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.0 scrub starts 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.0 scrub ok 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.3 scrub starts 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.3 scrub ok 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:53.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3037560740' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81692-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:53.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: osdmap e137: 8 total, 8 up, 8 in 2024-02-05T07:37:53.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.1 scrub starts 2024-02-05T07:37:53.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:52 smithi043 ceph-mon[41444]: 172.1 scrub ok 2024-02-05T07:37:53.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.7 scrub starts 2024-02-05T07:37:53.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.7 scrub ok 2024-02-05T07:37:53.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: pgmap v161: 460 pgs: 32 creating+peering, 32 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-05T07:37:53.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.0 scrub starts 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.0 scrub ok 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.3 scrub starts 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.3 scrub ok 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3037560740' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81692-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: osdmap e137: 8 total, 8 up, 8 in 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.1 scrub starts 2024-02-05T07:37:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:52 smithi050 ceph-mon[33492]: 172.1 scrub ok 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[29879]: osdmap e138: 8 total, 8 up, 8 in 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/968197851' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[29879]: 172.6 scrub starts 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[29879]: 172.6 scrub ok 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[41444]: osdmap e138: 8 total, 8 up, 8 in 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/968197851' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:54.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:54.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[41444]: 172.6 scrub starts 2024-02-05T07:37:54.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:53 smithi043 ceph-mon[41444]: 172.6 scrub ok 2024-02-05T07:37:54.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:54.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:53 smithi050 ceph-mon[33492]: osdmap e138: 8 total, 8 up, 8 in 2024-02-05T07:37:54.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/968197851' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:54.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:54.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:53 smithi050 ceph-mon[33492]: 172.6 scrub starts 2024-02-05T07:37:54.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:53 smithi050 ceph-mon[33492]: 172.6 scrub ok 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[29879]: pgmap v164: 460 pgs: 64 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[29879]: osdmap e139: 8 total, 8 up, 8 in 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2793615335' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81692-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[41444]: pgmap v164: 460 pgs: 64 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[41444]: osdmap e139: 8 total, 8 up, 8 in 2024-02-05T07:37:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2793615335' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81692-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:54 smithi050 ceph-mon[33492]: pgmap v164: 460 pgs: 64 unknown, 396 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 54 KiB/s wr, 127 op/s 2024-02-05T07:37:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81716-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:54 smithi050 ceph-mon[33492]: osdmap e139: 8 total, 8 up, 8 in 2024-02-05T07:37:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2793615335' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81692-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2793615335' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81692-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[29879]: osdmap e140: 8 total, 8 up, 8 in 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2793615335' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81692-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[41444]: osdmap e140: 8 total, 8 up, 8 in 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2793615335' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81692-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:55 smithi050 ceph-mon[33492]: osdmap e140: 8 total, 8 up, 8 in 2024-02-05T07:37:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started api_snapshots: Running main() from gmock_main.cc 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (5515 ms) 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (6629 ms) 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3631 ms) 2024-02-05T07:37:56.984 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (3033 ms) 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (18808 ms total) 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (3963 ms) 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4322 ms) 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (4971 ms) 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (13256 ms total) 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-05T07:37:56.985 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3077 ms) 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2005 ms) 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2010 ms) 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2007 ms) 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (9099 ms total) 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4038 ms) 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-05T07:37:56.986 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3998 ms) 2024-02-05T07:37:56.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8036 ms total) 2024-02-05T07:37:56.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-02-05T07:37:56.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-05T07:37:56.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (73229 ms total) 2024-02-05T07:37:56.987 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: pgmap v167: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: osdmap e141: 8 total, 8 up, 8 in 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4041236547' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81716-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:57.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: pgmap v167: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:57.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:57.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:57.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:57.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: osdmap e141: 8 total, 8 up, 8 in 2024-02-05T07:37:57.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:57.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4041236547' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81716-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:57.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:57.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: pgmap v167: 420 pgs: 32 unknown, 388 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:57.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:57.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:37:57.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:57.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: osdmap e141: 8 total, 8 up, 8 in 2024-02-05T07:37:57.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1758291709' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:57.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4041236547' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81716-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:57.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]: dispatch 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4041236547' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81716-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: osdmap e142: 8 total, 8 up, 8 in 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2"}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2921824799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81692-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:58.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4041236547' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81716-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: osdmap e142: 8 total, 8 up, 8 in 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2"}]: dispatch 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2921824799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81692-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:58.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:57 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-05T07:37:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4041236547' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81716-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81996-15"}]': finished 2024-02-05T07:37:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: osdmap e142: 8 total, 8 up, 8 in 2024-02-05T07:37:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2"}]: dispatch 2024-02-05T07:37:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2921824799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81692-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:37:58.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:57 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: pgmap v170: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2921824799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81692-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: osdmap e143: 8 total, 8 up, 8 in 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/523108840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82081-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:59.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: pgmap v170: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2921824799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81692-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: osdmap e143: 8 total, 8 up, 8 in 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/523108840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82081-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:59.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: pgmap v170: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:37:59.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:37:59.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81909-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:59.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2921824799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81692-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:37:59.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: osdmap e143: 8 total, 8 up, 8 in 2024-02-05T07:37:59.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/523108840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82081-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:37:59.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T07:37:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:37:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/523108840' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82081-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: osdmap e144: 8 total, 8 up, 8 in 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3878513640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:00.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:00.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/523108840' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82081-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:00.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T07:38:00.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: osdmap e144: 8 total, 8 up, 8 in 2024-02-05T07:38:00.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:38:00.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:38:00.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3878513640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:00.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:37:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:00.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:00.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/523108840' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82081-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:00.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T07:38:00.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: osdmap e144: 8 total, 8 up, 8 in 2024-02-05T07:38:00.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:38:00.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T07:38:00.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3878513640' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:00.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:37:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: pgmap v173: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: osdmap e145: 8 total, 8 up, 8 in 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4134761953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:38:01.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: pgmap v173: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: osdmap e145: 8 total, 8 up, 8 in 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4134761953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:38:01.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:01.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:38:01.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: pgmap v173: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:01.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:01.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T07:38:01.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:01.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: osdmap e145: 8 total, 8 up, 8 in 2024-02-05T07:38:01.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1657564122' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:38:01.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4134761953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:01.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T07:38:01.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:02.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-05T07:38:02.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-05T07:38:02.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-05T07:38:02.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-05T07:38:02.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-05T07:38:02.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-05T07:38:02.009 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: seed 81909 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (400 ms) 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (15 ms) 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (10 ms) 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (14 ms) 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-02-05T07:38:02.010 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (6 ms) 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (12 ms) 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (4 ms) 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-05T07:38:02.011 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1414 ms) 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (339 ms) 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting, doing copies 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60640 ms) 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (3 ms) 2024-02-05T07:38:02.012 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-05T07:38:02.013 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4074 ms) 2024-02-05T07:38:02.013 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-05T07:38:02.013 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-05T07:38:02.013 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-05T07:38:02.013 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-05T07:38:02.013 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-05T07:38:02.013 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]': finished 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[29879]: osdmap e146: 8 total, 8 up, 8 in 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:02.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:02.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]': finished 2024-02-05T07:38:02.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:02.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:01 smithi043 ceph-mon[41444]: osdmap e146: 8 total, 8 up, 8 in 2024-02-05T07:38:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:01 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:01 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81909-1","app":"app1","key":"key1"}]': finished 2024-02-05T07:38:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81692-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:02.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:01 smithi050 ceph-mon[33492]: osdmap e146: 8 total, 8 up, 8 in 2024-02-05T07:38:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: pgmap v176: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:38:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: osdmap e147: 8 total, 8 up, 8 in 2024-02-05T07:38:03.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/257531348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.065 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: pgmap v176: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:38:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: osdmap e147: 8 total, 8 up, 8 in 2024-02-05T07:38:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/257531348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:03.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.066 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.067 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: pgmap v176: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:38:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: osdmap e147: 8 total, 8 up, 8 in 2024-02-05T07:38:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/257531348' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.075 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.076 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:03.076 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:03.076 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: osdmap e148: 8 total, 8 up, 8 in 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1694771540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:04.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: osdmap e148: 8 total, 8 up, 8 in 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1694771540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:04.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:04.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:04.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81716-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81909-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: osdmap e148: 8 total, 8 up, 8 in 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1694771540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:04.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[29879]: pgmap v179: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:05.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[29879]: osdmap e149: 8 total, 8 up, 8 in 2024-02-05T07:38:05.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[41444]: pgmap v179: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:05.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:05.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:05.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:04 smithi043 ceph-mon[41444]: osdmap e149: 8 total, 8 up, 8 in 2024-02-05T07:38:05.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:04 smithi050 ceph-mon[33492]: pgmap v179: 420 pgs: 64 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:05.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:05.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81692-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:05.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:04 smithi050 ceph-mon[33492]: osdmap e149: 8 total, 8 up, 8 in 2024-02-05T07:38:06.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:06.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:06.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:06.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[29879]: osdmap e150: 8 total, 8 up, 8 in 2024-02-05T07:38:06.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/302001662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:06.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:06.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:06.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:06.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:06.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[41444]: osdmap e150: 8 total, 8 up, 8 in 2024-02-05T07:38:06.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/302001662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:06.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:06.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:06.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81909-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:06.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82081-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:06.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:05 smithi050 ceph-mon[33492]: osdmap e150: 8 total, 8 up, 8 in 2024-02-05T07:38:06.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/302001662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:06.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: pgmap v182: 404 pgs: 48 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: osdmap e151: 8 total, 8 up, 8 in 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2151464176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81692-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[41444]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:07.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: pgmap v182: 404 pgs: 48 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: osdmap e151: 8 total, 8 up, 8 in 2024-02-05T07:38:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2151464176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81692-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:06 smithi043 ceph-mon[29879]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: pgmap v182: 404 pgs: 48 unknown, 356 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-02-05T07:38:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81716-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: osdmap e151: 8 total, 8 up, 8 in 2024-02-05T07:38:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2151464176' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81692-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:07.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:06 smithi050 ceph-mon[33492]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2151464176' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81692-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[29879]: osdmap e152: 8 total, 8 up, 8 in 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2151464176' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81692-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:08.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:08.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[41444]: osdmap e152: 8 total, 8 up, 8 in 2024-02-05T07:38:08.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:08.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:08.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:08.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2151464176' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81692-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:08.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:08.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:07 smithi050 ceph-mon[33492]: osdmap e152: 8 total, 8 up, 8 in 2024-02-05T07:38:08.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:08.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[29879]: pgmap v185: 436 pgs: 64 unknown, 16 creating+peering, 356 active+clean; 592 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[29879]: osdmap e153: 8 total, 8 up, 8 in 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3743778243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81716-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[41444]: pgmap v185: 436 pgs: 64 unknown, 16 creating+peering, 356 active+clean; 592 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[41444]: osdmap e153: 8 total, 8 up, 8 in 2024-02-05T07:38:09.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3743778243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81716-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:09.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:08 smithi050 ceph-mon[33492]: pgmap v185: 436 pgs: 64 unknown, 16 creating+peering, 356 active+clean; 592 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:09.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:09.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81909-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:08 smithi050 ceph-mon[33492]: osdmap e153: 8 total, 8 up, 8 in 2024-02-05T07:38:09.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3743778243' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81716-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:38:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: seed 82081 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (4862 ms) 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (5234 ms) 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (3631 ms) 2024-02-05T07:38:09.979 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (3032 ms) 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (5115 ms) 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (21874 ms total) 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4081 ms) 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4181 ms) 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5741 ms) 2024-02-05T07:38:09.980 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4014 ms) 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2058 ms) 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-1241-g164d3be4/rpm/el9/BUILD/ceph-19.0.0-1241-g164d3be4/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi043-82081-7 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17857 ms) 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (37932 ms total) 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-02-05T07:38:09.981 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5065 ms) 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5074 ms total) 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2966 ms) 2024-02-05T07:38:09.982 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3743778243' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81716-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[29879]: osdmap e154: 8 total, 8 up, 8 in 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/165872178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3743778243' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81716-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[41444]: osdmap e154: 8 total, 8 up, 8 in 2024-02-05T07:38:10.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/165872178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:10.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:10.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:10.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3743778243' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81716-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:10.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:09 smithi050 ceph-mon[33492]: osdmap e154: 8 total, 8 up, 8 in 2024-02-05T07:38:10.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/165872178' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:10.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[41444]: pgmap v188: 436 pgs: 64 unknown, 16 creating+peering, 356 active+clean; 592 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[41444]: osdmap e155: 8 total, 8 up, 8 in 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[29879]: pgmap v188: 436 pgs: 64 unknown, 16 creating+peering, 356 active+clean; 592 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:11.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:11.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[29879]: osdmap e155: 8 total, 8 up, 8 in 2024-02-05T07:38:11.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:11.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:11.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:38:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:38:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:10 smithi050 ceph-mon[33492]: pgmap v188: 436 pgs: 64 unknown, 16 creating+peering, 356 active+clean; 592 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81692-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:10 smithi050 ceph-mon[33492]: osdmap e155: 8 total, 8 up, 8 in 2024-02-05T07:38:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:11.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:11.959 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:11.959 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:11.960 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:11.960 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: osdmap e156: 8 total, 8 up, 8 in 2024-02-05T07:38:11.960 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1279255827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:11.960 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: osdmap e156: 8 total, 8 up, 8 in 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1279255827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: osdmap e156: 8 total, 8 up, 8 in 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1279255827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/814759305' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:12.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:12.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:12.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]: dispatch 2024-02-05T07:38:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: pgmap v191: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:38:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: osdmap e157: 8 total, 8 up, 8 in 2024-02-05T07:38:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3871215226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: pgmap v191: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: osdmap e157: 8 total, 8 up, 8 in 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3871215226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:12 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: pgmap v191: 396 pgs: 32 unknown, 364 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:38:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81716-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81909-24"}]': finished 2024-02-05T07:38:13.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: osdmap e157: 8 total, 8 up, 8 in 2024-02-05T07:38:13.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3871215226' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:13.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:13.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:12 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[29879]: osdmap e158: 8 total, 8 up, 8 in 2024-02-05T07:38:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/327706666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81909-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:14.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[41444]: osdmap e158: 8 total, 8 up, 8 in 2024-02-05T07:38:14.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/327706666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81909-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81692-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:13 smithi050 ceph-mon[33492]: osdmap e158: 8 total, 8 up, 8 in 2024-02-05T07:38:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/327706666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81909-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:15.005 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosS snapshots: Running main() from gmock_main.cc 2024-02-05T07:38:15.005 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] Global test environment set-up. 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (17185 ms) 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (6513 ms) 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (5141 ms) 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (6068 ms) 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (6951 ms) 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (41858 ms total) 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: 2024-02-05T07:38:15.006 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (4993 ms) 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6090 ms) 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8193 ms) 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (5724 ms) 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4012 ms) 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi043-83006-11. 2024-02-05T07:38:15.007 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: Waiting for snaps to purge. 2024-02-05T07:38:15.008 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20031 ms) 2024-02-05T07:38:15.008 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (49043 ms total) 2024-02-05T07:38:15.008 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: 2024-02-05T07:38:15.008 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] Global test environment tear-down 2024-02-05T07:38:15.008 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (90901 ms total) 2024-02-05T07:38:15.008 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ PASSED ] 11 tests. 2024-02-05T07:38:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[29879]: pgmap v194: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:38:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/327706666' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81909-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[29879]: osdmap e159: 8 total, 8 up, 8 in 2024-02-05T07:38:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2179171134' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[41444]: pgmap v194: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:38:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/327706666' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81909-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[41444]: osdmap e159: 8 total, 8 up, 8 in 2024-02-05T07:38:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2179171134' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:14 smithi050 ceph-mon[33492]: pgmap v194: 428 pgs: 64 unknown, 364 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:38:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/327706666' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81909-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:14 smithi050 ceph-mon[33492]: osdmap e159: 8 total, 8 up, 8 in 2024-02-05T07:38:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2179171134' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[41444]: osdmap e160: 8 total, 8 up, 8 in 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1239792439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81692-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[41444]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[29879]: osdmap e160: 8 total, 8 up, 8 in 2024-02-05T07:38:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1239792439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81692-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:15 smithi043 ceph-mon[29879]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81716-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:15 smithi050 ceph-mon[33492]: osdmap e160: 8 total, 8 up, 8 in 2024-02-05T07:38:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1239792439' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81692-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:15 smithi050 ceph-mon[33492]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: pgmap v197: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1239792439' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81692-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: osdmap e161: 8 total, 8 up, 8 in 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1081758169' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81909-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:38:17.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: pgmap v197: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1239792439' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81692-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: osdmap e161: 8 total, 8 up, 8 in 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1081758169' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81909-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:38:17.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:38:17.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:38:17.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:16 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: pgmap v197: 388 pgs: 64 unknown, 324 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1239792439' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81692-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: osdmap e161: 8 total, 8 up, 8 in 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2238264617' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]: dispatch 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1081758169' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81909-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:38:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:16 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: pgmap v199: 388 pgs: 32 creating+peering, 32 unknown, 324 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:38:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-05T07:38:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1081758169' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81909-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:18.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: osdmap e162: 8 total, 8 up, 8 in 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2002278471' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: pgmap v199: 388 pgs: 32 creating+peering, 32 unknown, 324 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1081758169' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81909-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: osdmap e162: 8 total, 8 up, 8 in 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2002278471' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:18.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: pgmap v199: 388 pgs: 32 creating+peering, 32 unknown, 324 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82081-16"}]': finished 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1081758169' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81909-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: osdmap e162: 8 total, 8 up, 8 in 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2002278471' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:18.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:19.312 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:38:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: osdmap e163: 8 total, 8 up, 8 in 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:19.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: osdmap e163: 8 total, 8 up, 8 in 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81716-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81692-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: osdmap e163: 8 total, 8 up, 8 in 2024-02-05T07:38:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:20.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[29879]: pgmap v202: 356 pgs: 32 unknown, 324 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:38:20.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:20.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[29879]: osdmap e164: 8 total, 8 up, 8 in 2024-02-05T07:38:20.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2165508924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:20.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[41444]: pgmap v202: 356 pgs: 32 unknown, 324 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:38:20.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:20.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[41444]: osdmap e164: 8 total, 8 up, 8 in 2024-02-05T07:38:20.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2165508924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:20.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:19 smithi050 ceph-mon[33492]: pgmap v202: 356 pgs: 32 unknown, 324 active+clean; 592 KiB data, 595 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:38:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:19 smithi050 ceph-mon[33492]: osdmap e164: 8 total, 8 up, 8 in 2024-02-05T07:38:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2165508924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:20.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:21.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:38:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-mon[33492]: osdmap e165: 8 total, 8 up, 8 in 2024-02-05T07:38:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3218342981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81716-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:20 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[29879]: osdmap e165: 8 total, 8 up, 8 in 2024-02-05T07:38:21.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3218342981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81716-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81692-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81909-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[41444]: osdmap e165: 8 total, 8 up, 8 in 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3218342981' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81716-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:21.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:20 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: pgmap v205: 404 pgs: 48 creating+peering, 356 active+clean; 600 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 2 op/s 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3218342981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81716-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: osdmap e166: 8 total, 8 up, 8 in 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: pgmap v205: 404 pgs: 48 creating+peering, 356 active+clean; 600 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 2 op/s 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3218342981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81716-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: osdmap e166: 8 total, 8 up, 8 in 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:22.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: pgmap v205: 404 pgs: 48 creating+peering, 356 active+clean; 600 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 2 op/s 2024-02-05T07:38:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3218342981' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81716-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:22.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: osdmap e166: 8 total, 8 up, 8 in 2024-02-05T07:38:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:22.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[29879]: osdmap e167: 8 total, 8 up, 8 in 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[41444]: osdmap e167: 8 total, 8 up, 8 in 2024-02-05T07:38:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:22 smithi050 ceph-mon[33492]: osdmap e167: 8 total, 8 up, 8 in 2024-02-05T07:38:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:23.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: Running main() from gmock_main.cc 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment set-up. 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.TooBig (10382 ms) 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (7248 ms) 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-05T07:38:24.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (4415 ms) 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (4207 ms) 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3038 ms) 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3977 ms) 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3100 ms) 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3013 ms) 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2922 ms) 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-05T07:38:24.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3092 ms) 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2967 ms) 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3024 ms) 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3007 ms) 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2908 ms) 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.Flush (3014 ms) 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2994 ms) 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-05T07:38:24.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3006 ms) 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2994 ms) 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3021 ms) 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3000 ms) 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3015 ms) 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3006 ms) 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2963 ms) 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-05T07:38:24.043 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3014 ms) 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3011 ms) 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (3009 ms) 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 26 tests from LibRadosAio (93348 ms total) 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-05T07:38:24.044 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: pgmap v208: 332 pgs: 8 creating+peering, 324 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: osdmap e168: 8 total, 8 up, 8 in 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2193294922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: pgmap v208: 332 pgs: 8 creating+peering, 324 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: osdmap e168: 8 total, 8 up, 8 in 2024-02-05T07:38:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2193294922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: pgmap v208: 332 pgs: 8 creating+peering, 324 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2767183924' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: osdmap e168: 8 total, 8 up, 8 in 2024-02-05T07:38:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]: dispatch 2024-02-05T07:38:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2193294922' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: osdmap e169: 8 total, 8 up, 8 in 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: osdmap e169: 8 total, 8 up, 8 in 2024-02-05T07:38:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:25.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81909-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81692-27"}]': finished 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81716-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: osdmap e169: 8 total, 8 up, 8 in 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: pgmap v211: 372 pgs: 40 unknown, 8 creating+peering, 324 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: osdmap e170: 8 total, 8 up, 8 in 2024-02-05T07:38:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:38:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: pgmap v211: 372 pgs: 40 unknown, 8 creating+peering, 324 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: osdmap e170: 8 total, 8 up, 8 in 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:38:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:26.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:25 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: pgmap v211: 372 pgs: 40 unknown, 8 creating+peering, 324 active+clean; 592 KiB data, 587 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81692-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: osdmap e170: 8 total, 8 up, 8 in 2024-02-05T07:38:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:38:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]: dispatch 2024-02-05T07:38:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:25 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]': finished 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: osdmap e171: 8 total, 8 up, 8 in 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2811186078' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]': finished 2024-02-05T07:38:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: osdmap e171: 8 total, 8 up, 8 in 2024-02-05T07:38:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2811186078' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-13"}]': finished 2024-02-05T07:38:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: osdmap e171: 8 total, 8 up, 8 in 2024-02-05T07:38:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2811186078' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: (66953 ms total) 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (102 ms) 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (102 ms total) 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-02-05T07:38:28.053 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (39 ms) 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (2 ms) 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (41 ms total) 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (43 ms) 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (2 ms) 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (45 ms total) 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-05T07:38:28.054 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (140 ms) 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (2 ms) 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (142 ms total) 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1031 ms) 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1031 ms total) 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (104299 ms total) 2024-02-05T07:38:28.055 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: pgmap v214: 364 pgs: 32 unknown, 332 active+clean; 609 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: osdmap e172: 8 total, 8 up, 8 in 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:28.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: pgmap v214: 364 pgs: 32 unknown, 332 active+clean; 609 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: osdmap e172: 8 total, 8 up, 8 in 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:28.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:27 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:28.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: pgmap v214: 364 pgs: 32 unknown, 332 active+clean; 609 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 10 KiB/s wr, 23 op/s 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81692-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81716-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: osdmap e172: 8 total, 8 up, 8 in 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1442042412' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]: dispatch 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:28.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:27 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:29.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:38:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:38:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[29879]: osdmap e173: 8 total, 8 up, 8 in 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[41444]: osdmap e173: 8 total, 8 up, 8 in 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81909-36"}]': finished 2024-02-05T07:38:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:28 smithi050 ceph-mon[33492]: osdmap e173: 8 total, 8 up, 8 in 2024-02-05T07:38:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[29879]: pgmap v217: 340 pgs: 40 unknown, 300 active+clean; 609 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[29879]: osdmap e174: 8 total, 8 up, 8 in 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2289526091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81716-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[41444]: pgmap v217: 340 pgs: 40 unknown, 300 active+clean; 609 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-05T07:38:30.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[41444]: osdmap e174: 8 total, 8 up, 8 in 2024-02-05T07:38:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2289526091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81716-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:30.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:29 smithi050 ceph-mon[33492]: pgmap v217: 340 pgs: 40 unknown, 300 active+clean; 609 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-02-05T07:38:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:29 smithi050 ceph-mon[33492]: osdmap e174: 8 total, 8 up, 8 in 2024-02-05T07:38:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2289526091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81716-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:31.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2289526091' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81716-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: osdmap e175: 8 total, 8 up, 8 in 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:31.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:31.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: osdmap e176: 8 total, 8 up, 8 in 2024-02-05T07:38:31.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:31.074 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2289526091' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81716-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: osdmap e175: 8 total, 8 up, 8 in 2024-02-05T07:38:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:31.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: osdmap e176: 8 total, 8 up, 8 in 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2289526091' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81716-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:31.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: osdmap e175: 8 total, 8 up, 8 in 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1903739752' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]: dispatch 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81692-28"}]': finished 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: osdmap e176: 8 total, 8 up, 8 in 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:31.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: pgmap v220: 364 pgs: 364 active+clean; 618 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81692-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]': finished 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81692-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: osdmap e177: 8 total, 8 up, 8 in 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81692-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2442997520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81716-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:32.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: pgmap v220: 364 pgs: 364 active+clean; 618 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81692-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]': finished 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81692-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: osdmap e177: 8 total, 8 up, 8 in 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81692-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2442997520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81716-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:32.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:32.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: pgmap v220: 364 pgs: 364 active+clean; 618 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81692-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-15"}]': finished 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81692-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: osdmap e177: 8 total, 8 up, 8 in 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81692-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]: dispatch 2024-02-05T07:38:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2442997520' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81716-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:32.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:32.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:32.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:33 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]': finished 2024-02-05T07:38:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:33 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]': finished 2024-02-05T07:38:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:33 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-15", "mode": "writeback"}]': finished 2024-02-05T07:38:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: pgmap v223: 364 pgs: 32 unknown, 332 active+clean; 617 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-02-05T07:38:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2442997520' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81716-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:38:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: osdmap e178: 8 total, 8 up, 8 in 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81692-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: osdmap e179: 8 total, 8 up, 8 in 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: pgmap v223: 364 pgs: 32 unknown, 332 active+clean; 617 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2442997520' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81716-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: osdmap e178: 8 total, 8 up, 8 in 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81692-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: osdmap e179: 8 total, 8 up, 8 in 2024-02-05T07:38:34.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: pgmap v223: 364 pgs: 32 unknown, 332 active+clean; 617 KiB data, 594 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2442997520' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81716-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: osdmap e178: 8 total, 8 up, 8 in 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81692-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: osdmap e179: 8 total, 8 up, 8 in 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1519203356' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]: dispatch 2024-02-05T07:38:34.988 INFO:tasks.workunit.client.0.smithi043.stdout:napshotsECPP.SnapRemovePP (2007 ms) 2024-02-05T07:38:34.988 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2007 ms) 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2008 ms) 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8988 ms total) 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4053 ms) 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3975 ms) 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-05T07:38:34.989 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3933 ms) 2024-02-05T07:38:34.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11961 ms total) 2024-02-05T07:38:34.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-02-05T07:38:34.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-05T07:38:34.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (111175 ms total) 2024-02-05T07:38:34.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-05T07:38:34.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-02-05T07:38:34.990 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-02-05T07:38:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: pgmap v226: 332 pgs: 8 unknown, 324 active+clean; 585 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: osdmap e180: 8 total, 8 up, 8 in 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/247634975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: pgmap v226: 332 pgs: 8 unknown, 324 active+clean; 585 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:35.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:35.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: osdmap e180: 8 total, 8 up, 8 in 2024-02-05T07:38:35.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:35.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/247634975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:35.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:35.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: pgmap v226: 332 pgs: 8 unknown, 324 active+clean; 585 KiB data, 594 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82081-21"}]': finished 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: osdmap e180: 8 total, 8 up, 8 in 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/247634975' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]: dispatch 2024-02-05T07:38:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]': finished 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[29879]: osdmap e181: 8 total, 8 up, 8 in 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]': finished 2024-02-05T07:38:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[41444]: osdmap e181: 8 total, 8 up, 8 in 2024-02-05T07:38:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:36 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:36 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-15"}]': finished 2024-02-05T07:38:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81716-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:36 smithi050 ceph-mon[33492]: osdmap e181: 8 total, 8 up, 8 in 2024-02-05T07:38:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:36.991 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-02-05T07:38:36.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-02-05T07:38:36.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-05T07:38:36.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-02-05T07:38:36.992 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-05T07:38:36.993 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (10384 ms) 2024-02-05T07:38:36.993 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-05T07:38:36.993 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (23872 ms) 2024-02-05T07:38:36.993 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-05T07:38:36.993 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6139 ms) 2024-02-05T07:38:36.993 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-05T07:38:36.993 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3982 ms) 2024-02-05T07:38:36.994 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-05T07:38:36.994 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (2921 ms) 2024-02-05T07:38:36.994 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-05T07:38:36.994 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3071 ms) 2024-02-05T07:38:36.995 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-05T07:38:36.995 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3048 ms) 2024-02-05T07:38:36.995 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-05T07:38:36.995 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (2975 ms) 2024-02-05T07:38:36.995 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-05T07:38:36.995 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2909 ms) 2024-02-05T07:38:36.996 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-05T07:38:36.996 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3000 ms) 2024-02-05T07:38:36.996 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-05T07:38:36.996 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3009 ms) 2024-02-05T07:38:36.996 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-05T07:38:36.997 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2992 ms) 2024-02-05T07:38:36.997 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-05T07:38:36.997 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3008 ms) 2024-02-05T07:38:36.998 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-05T07:38:36.998 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3023 ms) 2024-02-05T07:38:36.999 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-05T07:38:36.999 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (2998 ms) 2024-02-05T07:38:36.999 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-05T07:38:36.999 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3527 ms) 2024-02-05T07:38:37.000 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-05T07:38:37.000 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2453 ms) 2024-02-05T07:38:37.000 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-05T07:38:37.000 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3010 ms) 2024-02-05T07:38:37.000 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-05T07:38:37.000 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3007 ms) 2024-02-05T07:38:37.000 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-05T07:38:37.001 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3015 ms) 2024-02-05T07:38:37.001 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-05T07:38:37.001 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3515 ms) 2024-02-05T07:38:37.001 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-05T07:38:37.001 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (2503 ms) 2024-02-05T07:38:37.001 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-05T07:38:37.001 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3014 ms) 2024-02-05T07:38:37.002 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-05T07:38:37.002 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3010 ms) 2024-02-05T07:38:37.002 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-05T07:38:37.002 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2930 ms) 2024-02-05T07:38:37.002 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-05T07:38:37.002 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3009 ms) 2024-02-05T07:38:37.002 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-05T07:38:38.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:37 smithi043 ceph-mon[29879]: pgmap v229: 356 pgs: 32 creating+peering, 324 active+clean; 585 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:38.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:38.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:37 smithi043 ceph-mon[29879]: osdmap e182: 8 total, 8 up, 8 in 2024-02-05T07:38:38.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:38.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:38.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:38 smithi043 ceph-mon[41444]: pgmap v229: 356 pgs: 32 creating+peering, 324 active+clean; 585 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:38.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:38.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:38 smithi043 ceph-mon[41444]: osdmap e182: 8 total, 8 up, 8 in 2024-02-05T07:38:38.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:38.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:38.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:37 smithi050 ceph-mon[33492]: pgmap v229: 356 pgs: 32 creating+peering, 324 active+clean; 585 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:38.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:38.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:37 smithi050 ceph-mon[33492]: osdmap e182: 8 total, 8 up, 8 in 2024-02-05T07:38:38.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:38.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]: dispatch 2024-02-05T07:38:39.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:38:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:38:39.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:39.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: osdmap e183: 8 total, 8 up, 8 in 2024-02-05T07:38:39.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1154446544' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81692-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81692-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: osdmap e184: 8 total, 8 up, 8 in 2024-02-05T07:38:39.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81692-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: osdmap e183: 8 total, 8 up, 8 in 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1154446544' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81692-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:39.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:39.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:39.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81692-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:39.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: osdmap e184: 8 total, 8 up, 8 in 2024-02-05T07:38:39.316 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81692-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2175254705' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81692-29"}]': finished 2024-02-05T07:38:39.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: osdmap e183: 8 total, 8 up, 8 in 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1154446544' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81692-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81716-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81692-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: osdmap e184: 8 total, 8 up, 8 in 2024-02-05T07:38:39.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81692-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[29879]: pgmap v232: 356 pgs: 64 unknown, 292 active+clean; 585 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[41444]: pgmap v232: 356 pgs: 64 unknown, 292 active+clean; 585 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:40 smithi050 ceph-mon[33492]: pgmap v232: 356 pgs: 64 unknown, 292 active+clean; 585 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:41.251 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:38:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:38:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: osdmap e185: 8 total, 8 up, 8 in 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: pgmap v235: 324 pgs: 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81692-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]': finished 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: osdmap e186: 8 total, 8 up, 8 in 2024-02-05T07:38:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1188212590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: osdmap e185: 8 total, 8 up, 8 in 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: pgmap v235: 324 pgs: 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81692-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]': finished 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: osdmap e186: 8 total, 8 up, 8 in 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]: dispatch 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1188212590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]: dispatch 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:41.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:41.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: osdmap e185: 8 total, 8 up, 8 in 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: pgmap v235: 324 pgs: 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81692-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-17"}]': finished 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-05T07:38:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: osdmap e186: 8 total, 8 up, 8 in 2024-02-05T07:38:41.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]: dispatch 2024-02-05T07:38:41.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1188212590' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:41.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]: dispatch 2024-02-05T07:38:41.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:41.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]': finished 2024-02-05T07:38:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: osdmap e187: 8 total, 8 up, 8 in 2024-02-05T07:38:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:42.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:42.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]': finished 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: osdmap e187: 8 total, 8 up, 8 in 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:42.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-17", "mode": "writeback"}]': finished 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81716-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: osdmap e187: 8 total, 8 up, 8 in 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]: dispatch 2024-02-05T07:38:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:43.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: pgmap v238: 364 pgs: 40 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:38:43.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:43.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:38:43.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: osdmap e188: 8 total, 8 up, 8 in 2024-02-05T07:38:43.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:43.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:43.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:43.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:43.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:43.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:43.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:43.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:43.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:43.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: pgmap v238: 364 pgs: 40 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:38:43.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:43.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:38:43.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: osdmap e188: 8 total, 8 up, 8 in 2024-02-05T07:38:43.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:43.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:43.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:43.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:43.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:43.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:43.818 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: pgmap v238: 364 pgs: 40 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:38:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pg_num","val":"11"}]': finished 2024-02-05T07:38:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: osdmap e188: 8 total, 8 up, 8 in 2024-02-05T07:38:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:43.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:43.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:43.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:43.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]: dispatch 2024-02-05T07:38:43.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-05T07:38:43.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]': finished 2024-02-05T07:38:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:38:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: osdmap e189: 8 total, 8 up, 8 in 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3183948969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81716-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]': finished 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: osdmap e189: 8 total, 8 up, 8 in 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3183948969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81716-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-17"}]': finished 2024-02-05T07:38:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81789-1","var":"pgp_num","val":"11"}]': finished 2024-02-05T07:38:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: osdmap e189: 8 total, 8 up, 8 in 2024-02-05T07:38:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]: dispatch 2024-02-05T07:38:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3183948969' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81716-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: pgmap v241: 356 pgs: 32 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3183948969' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81716-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: osdmap e190: 8 total, 8 up, 8 in 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: pgmap v241: 356 pgs: 32 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3183948969' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81716-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: osdmap e190: 8 total, 8 up, 8 in 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:45 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: pgmap v241: 356 pgs: 32 unknown, 324 active+clean; 585 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1471090' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81692-30"}]': finished 2024-02-05T07:38:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3183948969' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81716-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: osdmap e190: 8 total, 8 up, 8 in 2024-02-05T07:38:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:45 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: osdmap e191: 8 total, 8 up, 8 in 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: osdmap e191: 8 total, 8 up, 8 in 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:47.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:46 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81692-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: osdmap e191: 8 total, 8 up, 8 in 2024-02-05T07:38:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:46 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: pgmap v244: 324 pgs: 32 unknown, 292 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: osdmap e192: 8 total, 8 up, 8 in 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/63754191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: pgmap v244: 324 pgs: 32 unknown, 292 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: osdmap e192: 8 total, 8 up, 8 in 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/63754191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: pgmap v244: 324 pgs: 32 unknown, 292 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-05T07:38:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-05T07:38:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: osdmap e192: 8 total, 8 up, 8 in 2024-02-05T07:38:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/63754191' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:49.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:38:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:38:49.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:49.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:49.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: osdmap e193: 8 total, 8 up, 8 in 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/63754191' entity='client.admin' cmd=[{ 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "prefix": "osd pool set", 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "var": "eio", 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "val": "true" 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: }]: dispatch 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-05T07:38:49.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "prefix": "osd pool set", 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "var": "eio", 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: "val": "true" 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[29879]: }]: dispatch 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: osdmap e193: 8 total, 8 up, 8 in 2024-02-05T07:38:49.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/63754191' entity='client.admin' cmd=[{ 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "prefix": "osd pool set", 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "var": "eio", 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "val": "true" 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: }]: dispatch 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "prefix": "osd pool set", 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "var": "eio", 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: "val": "true" 2024-02-05T07:38:49.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:48 smithi043 ceph-mon[41444]: }]: dispatch 2024-02-05T07:38:49.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:49.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81692-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:49.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81716-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: osdmap e193: 8 total, 8 up, 8 in 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/63754191' entity='client.admin' cmd=[{ 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "prefix": "osd pool set", 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "var": "eio", 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "val": "true" 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: }]: dispatch 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{ 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "prefix": "osd pool set", 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:49.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "var": "eio", 2024-02-05T07:38:49.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: "val": "true" 2024-02-05T07:38:49.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:48 smithi050 ceph-mon[33492]: }]: dispatch 2024-02-05T07:38:50.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: pgmap v247: 364 pgs: 72 unknown, 292 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-05T07:38:50.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:50.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]': finished 2024-02-05T07:38:50.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-05T07:38:50.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: "prefix": "osd pool set", 2024-02-05T07:38:50.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: "var": "eio", 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: "val": "true" 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: }]': finished 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]: dispatch 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: osdmap e194: 8 total, 8 up, 8 in 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]: dispatch 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: pgmap v247: 364 pgs: 72 unknown, 292 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]': finished 2024-02-05T07:38:50.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: "prefix": "osd pool set", 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: "var": "eio", 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: "val": "true" 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: }]': finished 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]: dispatch 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: osdmap e194: 8 total, 8 up, 8 in 2024-02-05T07:38:50.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]: dispatch 2024-02-05T07:38:50.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: pgmap v247: 364 pgs: 72 unknown, 292 active+clean; 585 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-05T07:38:50.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:50.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-19"}]': finished 2024-02-05T07:38:50.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{ 2024-02-05T07:38:50.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: "prefix": "osd pool set", 2024-02-05T07:38:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: "pool": "PoolEIOFlag_smithi043-81716-33", 2024-02-05T07:38:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: "var": "eio", 2024-02-05T07:38:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: "val": "true" 2024-02-05T07:38:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: }]': finished 2024-02-05T07:38:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]: dispatch 2024-02-05T07:38:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: osdmap e194: 8 total, 8 up, 8 in 2024-02-05T07:38:50.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]: dispatch 2024-02-05T07:38:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]': finished 2024-02-05T07:38:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: osdmap e195: 8 total, 8 up, 8 in 2024-02-05T07:38:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:51.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]': finished 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: osdmap e195: 8 total, 8 up, 8 in 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:51.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-05T07:38:51.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:50 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-05T07:38:51.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:38:51.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:51.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:51.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-19", "mode": "writeback"}]': finished 2024-02-05T07:38:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: osdmap e195: 8 total, 8 up, 8 in 2024-02-05T07:38:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-05T07:38:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:50 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-05T07:38:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: pgmap v250: 323 pgs: 323 active+clean; 585 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:38:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: osdmap e196: 8 total, 8 up, 8 in 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1485134697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81716-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: pgmap v250: 323 pgs: 323 active+clean; 585 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: osdmap e196: 8 total, 8 up, 8 in 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:52.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1485134697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81716-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: pgmap v250: 323 pgs: 323 active+clean; 585 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:38:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:38:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81789-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1500624456' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: osdmap e196: 8 total, 8 up, 8 in 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]: dispatch 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]: dispatch 2024-02-05T07:38:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1485134697' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81716-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]': finished 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1485134697' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81716-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: osdmap e197: 8 total, 8 up, 8 in 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]': finished 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1485134697' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81716-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: osdmap e197: 8 total, 8 up, 8 in 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:53.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3857014201' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-05T07:38:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:38:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81692-31"}]': finished 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-19"}]': finished 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1485134697' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81716-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: osdmap e197: 8 total, 8 up, 8 in 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: pgmap v253: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: osdmap e198: 8 total, 8 up, 8 in 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: pgmap v253: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: osdmap e198: 8 total, 8 up, 8 in 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: pgmap v253: 355 pgs: 32 unknown, 323 active+clean; 585 KiB data, 597 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:38:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81692-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: osdmap e198: 8 total, 8 up, 8 in 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: osdmap e199: 8 total, 8 up, 8 in 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2139464052' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: osdmap e199: 8 total, 8 up, 8 in 2024-02-05T07:38:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2139464052' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81789-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:38:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: osdmap e199: 8 total, 8 up, 8 in 2024-02-05T07:38:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2139464052' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[29879]: pgmap v256: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[29879]: osdmap e200: 8 total, 8 up, 8 in 2024-02-05T07:38:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:56.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[41444]: pgmap v256: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:56.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:56.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:56.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:56.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:56.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[41444]: osdmap e200: 8 total, 8 up, 8 in 2024-02-05T07:38:56.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:56 smithi050 ceph-mon[33492]: pgmap v256: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 597 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:38:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81692-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81716-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:56.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:56.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:56 smithi050 ceph-mon[33492]: osdmap e200: 8 total, 8 up, 8 in 2024-02-05T07:38:56.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:38:57.110 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAiry=15 expected=15 2024-02-05T07:38:57.110 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:6cac518f:::0:head 2024-02-05T07:38:57.110 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:6cac518f:::0:head expected=15:6cac518f:::0:head 2024-02-05T07:38:57.110 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:6cac518f:::0:head -> 0 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=0 expected=0 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:62a1935d:::14:head 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:62a1935d:::14:head expected=15:62a1935d:::14:head 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:62a1935d:::14:head -> 14 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=14 expected=14 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:f905c69b:::2:head 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:f905c69b:::2:head expected=15:f905c69b:::2:head 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:f905c69b:::2:head -> 2 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=2 expected=2 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:e960b815:::9:head 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:e960b815:::9:head expected=15:e960b815:::9:head 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:e960b815:::9:head -> 9 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=9 expected=9 2024-02-05T07:38:57.111 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:de5d7c5f:::12:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:de5d7c5f:::12:head expected=15:de5d7c5f:::12:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:de5d7c5f:::12:head -> 12 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=12 expected=12 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:d83876eb:::4:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:d83876eb:::4:head expected=15:d83876eb:::4:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:d83876eb:::4:head -> 4 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=4 expected=4 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:02547ec2:::1:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:02547ec2:::1:head expected=15:02547ec2:::1:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:02547ec2:::1:head -> 1 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=1 expected=1 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 15:52ea6a34:::10:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=15:52ea6a34:::10:head expected=15:52ea6a34:::10:head 2024-02-05T07:38:57.112 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 15:52ea6a34:::10:head -> 10 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=10 expected=10 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (182 ms) 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (107203 ms) 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 0/5 -> MIN 15:33333333::::head 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 1/5 -> 15:33333333::::head 15:66666666::::head 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 2/5 -> 15:66666666::::head 15:99999999::::head 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 3/5 -> 15:99999999::::head 15:cccccccc::::head 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 4/5 -> 15:cccccccc::::head MAX 2024-02-05T07:38:57.113 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (10033 ms) 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList (118048 ms total) 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1035 ms) 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-02-05T07:38:57.114 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-02-05T07:38:57.115 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (33 ms) 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[29879]: osdmap e201: 8 total, 8 up, 8 in 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[41444]: osdmap e201: 8 total, 8 up, 8 in 2024-02-05T07:38:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:38:57.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81789-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:38:57.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:38:57.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:38:57.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:57 smithi050 ceph-mon[33492]: osdmap e201: 8 total, 8 up, 8 in 2024-02-05T07:38:57.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:38:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: pgmap v259: 308 pgs: 8 creating+peering, 8 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:38:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]': finished 2024-02-05T07:38:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]: dispatch 2024-02-05T07:38:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: osdmap e202: 8 total, 8 up, 8 in 2024-02-05T07:38:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1278972828' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]: dispatch 2024-02-05T07:38:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:58.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:58.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:58.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: pgmap v259: 308 pgs: 8 creating+peering, 8 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:38:58.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]': finished 2024-02-05T07:38:58.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]: dispatch 2024-02-05T07:38:58.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: osdmap e202: 8 total, 8 up, 8 in 2024-02-05T07:38:58.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1278972828' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:58.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:58.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]: dispatch 2024-02-05T07:38:58.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:58.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:58.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:58.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: pgmap v259: 308 pgs: 8 creating+peering, 8 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:38:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-21"}]': finished 2024-02-05T07:38:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]: dispatch 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: osdmap e202: 8 total, 8 up, 8 in 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1278972828' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]: dispatch 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:38:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:38:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]': finished 2024-02-05T07:38:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: osdmap e203: 8 total, 8 up, 8 in 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:38:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:38:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]': finished 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: osdmap e203: 8 total, 8 up, 8 in 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-21", "mode": "writeback"}]': finished 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: osdmap e203: 8 total, 8 up, 8 in 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1194082735' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]: dispatch 2024-02-05T07:38:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:38:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:38:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81692-32"}]': finished 2024-02-05T07:38:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:38:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: pgmap v262: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: osdmap e204: 8 total, 8 up, 8 in 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2890653873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:39:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81692-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]': finished 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: pgmap v262: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: osdmap e204: 8 total, 8 up, 8 in 2024-02-05T07:39:00.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2890653873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81692-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:00.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]': finished 2024-02-05T07:39:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: pgmap v262: 324 pgs: 32 unknown, 292 active+clean; 583 KiB data, 569 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: osdmap e204: 8 total, 8 up, 8 in 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/721510889' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2890653873' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81692-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-21"}]': finished 2024-02-05T07:39:01.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:39:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81692-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[29879]: osdmap e205: 8 total, 8 up, 8 in 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81692-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81692-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[41444]: osdmap e205: 8 total, 8 up, 8 in 2024-02-05T07:39:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81692-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81789-2"}]': finished 2024-02-05T07:39:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81692-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:01 smithi050 ceph-mon[33492]: osdmap e205: 8 total, 8 up, 8 in 2024-02-05T07:39:01.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81692-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: pgmap v265: 356 pgs: 356 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:39:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: osdmap e206: 8 total, 8 up, 8 in 2024-02-05T07:39:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3392319720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81789-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81692-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81789-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: osdmap e207: 8 total, 8 up, 8 in 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81789-3","pool2":"test-rados-api-smithi043-81789-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: pgmap v265: 356 pgs: 356 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:39:02.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: osdmap e206: 8 total, 8 up, 8 in 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3392319720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81789-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81692-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81789-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: osdmap e207: 8 total, 8 up, 8 in 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81789-3","pool2":"test-rados-api-smithi043-81789-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:39:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: pgmap v265: 356 pgs: 356 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:39:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: osdmap e206: 8 total, 8 up, 8 in 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3392319720' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81789-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81692-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81789-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81716-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: osdmap e207: 8 total, 8 up, 8 in 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:02.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81789-3","pool2":"test-rados-api-smithi043-81789-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-02-05T07:39:03.087 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (43 ms) 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC (1111 ms total) 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-05T07:39:03.088 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-05T07:39:03.089 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3009 ms) 2024-02-05T07:39:03.089 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP (3009 ms total) 2024-02-05T07:39:03.089 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-02-05T07:39:03.089 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment tear-down 2024-02-05T07:39:03.089 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] 11 tests from 3 test suites ran. (139382 ms total) 2024-02-05T07:39:03.089 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ PASSED ] 11 tests. 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[29879]: pgmap v268: 428 pgs: 104 unknown, 324 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81789-3","pool2":"test-rados-api-smithi043-81789-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[29879]: osdmap e208: 8 total, 8 up, 8 in 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[41444]: pgmap v268: 428 pgs: 104 unknown, 324 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81789-3","pool2":"test-rados-api-smithi043-81789-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-05T07:39:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:04 smithi043 ceph-mon[41444]: osdmap e208: 8 total, 8 up, 8 in 2024-02-05T07:39:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:04 smithi050 ceph-mon[33492]: pgmap v268: 428 pgs: 104 unknown, 324 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:39:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3055579840' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81789-3","pool2":"test-rados-api-smithi043-81789-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-05T07:39:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:04 smithi050 ceph-mon[33492]: osdmap e208: 8 total, 8 up, 8 in 2024-02-05T07:39:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[29879]: osdmap e209: 8 total, 8 up, 8 in 2024-02-05T07:39:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[41444]: osdmap e209: 8 total, 8 up, 8 in 2024-02-05T07:39:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:05 smithi050 ceph-mon[33492]: osdmap e209: 8 total, 8 up, 8 in 2024-02-05T07:39:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: pgmap v271: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: osdmap e210: 8 total, 8 up, 8 in 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: pgmap v271: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: osdmap e210: 8 total, 8 up, 8 in 2024-02-05T07:39:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:06.466 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: pgmap v271: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 578 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: osdmap e210: 8 total, 8 up, 8 in 2024-02-05T07:39:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]: dispatch 2024-02-05T07:39:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]': finished 2024-02-05T07:39:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]: dispatch 2024-02-05T07:39:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: osdmap e211: 8 total, 8 up, 8 in 2024-02-05T07:39:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4089371066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]': finished 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]: dispatch 2024-02-05T07:39:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: osdmap e211: 8 total, 8 up, 8 in 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4089371066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:07.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1407610427' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81692-33"}]': finished 2024-02-05T07:39:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-23"}]': finished 2024-02-05T07:39:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]: dispatch 2024-02-05T07:39:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: osdmap e211: 8 total, 8 up, 8 in 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4089371066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: 0x55567c1a12c0 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0e20 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a0f30 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1040 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1150 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1890 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a13f0 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1500 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1610 2024-02-05T07:39:08.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1720 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1e40 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a19a0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1ab0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1bc0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1d10 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a23f0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a1f50 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a2060 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a2190 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a22a0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a29e0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a2540 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a2670 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a27d0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a2af0 2024-02-05T07:39:08.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a2fa0 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a30b0 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e360 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e490 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a36a0 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3200 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3350 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3460 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3570 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c19e5a0 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3810 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3c80 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3db0 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4360 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3920 2024-02-05T07:39:08.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3a30 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3b60 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a44b0 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a3ec0 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4020 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4150 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a45e0 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4a70 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4b80 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a46f0 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4870 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5150 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4cb0 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4de0 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a4ef0 2024-02-05T07:39:08.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5000 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5700 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5260 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a53b0 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a54c0 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a55f0 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5cb0 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5810 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5960 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5a70 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5ba0 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6280 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5de0 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a5ef0 2024-02-05T07:39:08.404 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6020 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6170 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6850 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a63b0 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a64e0 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6610 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6740 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6e20 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6980 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6ad0 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6c00 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6f50 2024-02-05T07:39:08.405 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a73e0 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: pgmap v274: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]': finished 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: osdmap e212: 8 total, 8 up, 8 in 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: pgmap v274: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:39:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]': finished 2024-02-05T07:39:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: osdmap e212: 8 total, 8 up, 8 in 2024-02-05T07:39:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: pgmap v274: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:39:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-23", "mode": "writeback"}]': finished 2024-02-05T07:39:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81692-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: osdmap e212: 8 total, 8 up, 8 in 2024-02-05T07:39:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:39:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:39:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[41444]: osdmap e213: 8 total, 8 up, 8 in 2024-02-05T07:39:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1506778635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[29879]: osdmap e213: 8 total, 8 up, 8 in 2024-02-05T07:39:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1506778635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:09 smithi050 ceph-mon[33492]: osdmap e213: 8 total, 8 up, 8 in 2024-02-05T07:39:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1506778635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:09.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[41444]: pgmap v277: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:39:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[41444]: osdmap e214: 8 total, 8 up, 8 in 2024-02-05T07:39:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[29879]: pgmap v277: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:39:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[29879]: osdmap e214: 8 total, 8 up, 8 in 2024-02-05T07:39:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-mon[33492]: pgmap v277: 356 pgs: 64 unknown, 292 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:39:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81692-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-mon[33492]: osdmap e214: 8 total, 8 up, 8 in 2024-02-05T07:39:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]: dispatch 2024-02-05T07:39:11.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:39:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:39:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]': finished 2024-02-05T07:39:11.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[29879]: osdmap e215: 8 total, 8 up, 8 in 2024-02-05T07:39:11.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3931445622' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:11.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:11.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:11.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:11.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]': finished 2024-02-05T07:39:11.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[41444]: osdmap e215: 8 total, 8 up, 8 in 2024-02-05T07:39:11.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3931445622' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:11.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:11.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:11 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:11.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:11 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:11.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-23"}]': finished 2024-02-05T07:39:11.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:11 smithi050 ceph-mon[33492]: osdmap e215: 8 total, 8 up, 8 in 2024-02-05T07:39:11.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3931445622' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:11.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:11.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:11 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: pgmap v280: 396 pgs: 32 creating+peering, 364 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-05T07:39:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: osdmap e216: 8 total, 8 up, 8 in 2024-02-05T07:39:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: osdmap e217: 8 total, 8 up, 8 in 2024-02-05T07:39:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/55035589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: pgmap v280: 396 pgs: 32 creating+peering, 364 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-05T07:39:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: osdmap e216: 8 total, 8 up, 8 in 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: osdmap e217: 8 total, 8 up, 8 in 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/55035589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: pgmap v280: 396 pgs: 32 creating+peering, 364 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 255 B/s wr, 5 op/s 2024-02-05T07:39:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: osdmap e216: 8 total, 8 up, 8 in 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4108265593' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: osdmap e217: 8 total, 8 up, 8 in 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/55035589' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]: dispatch 2024-02-05T07:39:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: pgmap v283: 420 pgs: 64 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/55035589' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: osdmap e218: 8 total, 8 up, 8 in 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81692-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: pgmap v283: 420 pgs: 64 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/55035589' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: osdmap e218: 8 total, 8 up, 8 in 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81692-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: pgmap v283: 420 pgs: 64 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:39:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/55035589' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81692-34"}]': finished 2024-02-05T07:39:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: osdmap e218: 8 total, 8 up, 8 in 2024-02-05T07:39:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81692-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81692-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[29879]: osdmap e219: 8 total, 8 up, 8 in 2024-02-05T07:39:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81692-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3986813696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81692-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[41444]: osdmap e219: 8 total, 8 up, 8 in 2024-02-05T07:39:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81692-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3986813696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81692-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:15 smithi050 ceph-mon[33492]: osdmap e219: 8 total, 8 up, 8 in 2024-02-05T07:39:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81692-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3986813696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:15.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:15.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: pgmap v286: 452 pgs: 96 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: osdmap e220: 8 total, 8 up, 8 in 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-81692-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]': finished 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: osdmap e221: 8 total, 8 up, 8 in 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]: dispatch 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]: dispatch 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: pgmap v286: 452 pgs: 96 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: osdmap e220: 8 total, 8 up, 8 in 2024-02-05T07:39:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:16.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-81692-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:16.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]': finished 2024-02-05T07:39:16.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: osdmap e221: 8 total, 8 up, 8 in 2024-02-05T07:39:16.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]: dispatch 2024-02-05T07:39:16.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]: dispatch 2024-02-05T07:39:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: pgmap v286: 452 pgs: 96 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 567 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81716-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: osdmap e220: 8 total, 8 up, 8 in 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-81692-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-25"}]': finished 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: osdmap e221: 8 total, 8 up, 8 in 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]: dispatch 2024-02-05T07:39:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]: dispatch 2024-02-05T07:39:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:39:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]': finished 2024-02-05T07:39:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[29879]: osdmap e222: 8 total, 8 up, 8 in 2024-02-05T07:39:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:39:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]': finished 2024-02-05T07:39:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[41444]: osdmap e222: 8 total, 8 up, 8 in 2024-02-05T07:39:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:17 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:39:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:17 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-25", "mode": "writeback"}]': finished 2024-02-05T07:39:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:17 smithi050 ceph-mon[33492]: osdmap e222: 8 total, 8 up, 8 in 2024-02-05T07:39:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: pgmap v289: 428 pgs: 8 unknown, 420 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:39:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:39:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:39:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: osdmap e223: 8 total, 8 up, 8 in 2024-02-05T07:39:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: pgmap v289: 428 pgs: 8 unknown, 420 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: osdmap e223: 8 total, 8 up, 8 in 2024-02-05T07:39:18.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:18.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:18.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: pgmap v289: 428 pgs: 8 unknown, 420 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:39:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: osdmap e223: 8 total, 8 up, 8 in 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]: dispatch 2024-02-05T07:39:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:39:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:39:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]': finished 2024-02-05T07:39:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[29879]: osdmap e224: 8 total, 8 up, 8 in 2024-02-05T07:39:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]': finished 2024-02-05T07:39:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[41444]: osdmap e224: 8 total, 8 up, 8 in 2024-02-05T07:39:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:19 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-25"}]': finished 2024-02-05T07:39:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:19 smithi050 ceph-mon[33492]: osdmap e224: 8 total, 8 up, 8 in 2024-02-05T07:39:19.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]: dispatch 2024-02-05T07:39:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[29879]: pgmap v292: 356 pgs: 356 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:39:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[29879]: osdmap e225: 8 total, 8 up, 8 in 2024-02-05T07:39:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[41444]: pgmap v292: 356 pgs: 356 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:39:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:20 smithi043 ceph-mon[41444]: osdmap e225: 8 total, 8 up, 8 in 2024-02-05T07:39:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:20 smithi050 ceph-mon[33492]: pgmap v292: 356 pgs: 356 active+clean; 584 KiB data, 568 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:39:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:20 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:20.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2457343149' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81692-35"}]': finished 2024-02-05T07:39:20.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:20 smithi050 ceph-mon[33492]: osdmap e225: 8 total, 8 up, 8 in 2024-02-05T07:39:21.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:39:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:39:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: pgmap v295: 260 pgs: 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: osdmap e226: 8 total, 8 up, 8 in 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/356853930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: pgmap v295: 260 pgs: 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: osdmap e226: 8 total, 8 up, 8 in 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/356853930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: pgmap v295: 260 pgs: 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81692-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: osdmap e226: 8 total, 8 up, 8 in 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/356853930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:22 smithi043 ceph-mon[29879]: osdmap e227: 8 total, 8 up, 8 in 2024-02-05T07:39:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:22 smithi043 ceph-mon[41444]: osdmap e227: 8 total, 8 up, 8 in 2024-02-05T07:39:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81716-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:22 smithi050 ceph-mon[33492]: osdmap e227: 8 total, 8 up, 8 in 2024-02-05T07:39:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[29879]: pgmap v298: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[29879]: osdmap e228: 8 total, 8 up, 8 in 2024-02-05T07:39:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:24.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[41444]: pgmap v298: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[41444]: osdmap e228: 8 total, 8 up, 8 in 2024-02-05T07:39:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:24.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:24 smithi050 ceph-mon[33492]: pgmap v298: 324 pgs: 64 unknown, 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81692-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:24 smithi050 ceph-mon[33492]: osdmap e228: 8 total, 8 up, 8 in 2024-02-05T07:39:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: osdmap e229: 8 total, 8 up, 8 in 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4148556428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81716-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]': finished 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4148556428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81716-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: osdmap e230: 8 total, 8 up, 8 in 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]: dispatch 2024-02-05T07:39:25.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]: dispatch 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: osdmap e229: 8 total, 8 up, 8 in 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4148556428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81716-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]': finished 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4148556428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81716-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: osdmap e230: 8 total, 8 up, 8 in 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]: dispatch 2024-02-05T07:39:25.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]: dispatch 2024-02-05T07:39:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: osdmap e229: 8 total, 8 up, 8 in 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4148556428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81716-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-27"}]': finished 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4148556428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81716-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: osdmap e230: 8 total, 8 up, 8 in 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]: dispatch 2024-02-05T07:39:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]: dispatch 2024-02-05T07:39:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: pgmap v301: 332 pgs: 72 unknown, 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]': finished 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: osdmap e231: 8 total, 8 up, 8 in 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: pgmap v301: 332 pgs: 72 unknown, 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]': finished 2024-02-05T07:39:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:26.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: osdmap e231: 8 total, 8 up, 8 in 2024-02-05T07:39:26.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: pgmap v301: 332 pgs: 72 unknown, 260 active+clean; 583 KiB data, 572 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-27", "mode": "writeback"}]': finished 2024-02-05T07:39:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: osdmap e231: 8 total, 8 up, 8 in 2024-02-05T07:39:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2271166172' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]: dispatch 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: osdmap e232: 8 total, 8 up, 8 in 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: osdmap e232: 8 total, 8 up, 8 in 2024-02-05T07:39:27.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81692-36"}]': finished 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: osdmap e232: 8 total, 8 up, 8 in 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: pgmap v304: 292 pgs: 292 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: pgmap v304: 292 pgs: 292 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: pgmap v304: 292 pgs: 292 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:39:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:39:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[29879]: osdmap e233: 8 total, 8 up, 8 in 2024-02-05T07:39:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81692-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:29.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:29.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:29.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[41444]: osdmap e233: 8 total, 8 up, 8 in 2024-02-05T07:39:29.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81692-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:29.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:29.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:29.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:29 smithi050 ceph-mon[33492]: osdmap e233: 8 total, 8 up, 8 in 2024-02-05T07:39:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81692-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]: dispatch 2024-02-05T07:39:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:29 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[29879]: pgmap v307: 292 pgs: 292 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:39:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]': finished 2024-02-05T07:39:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[29879]: osdmap e234: 8 total, 8 up, 8 in 2024-02-05T07:39:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81692-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[29879]: osdmap e235: 8 total, 8 up, 8 in 2024-02-05T07:39:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[41444]: pgmap v307: 292 pgs: 292 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:39:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]': finished 2024-02-05T07:39:30.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[41444]: osdmap e234: 8 total, 8 up, 8 in 2024-02-05T07:39:30.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81692-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:30.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:30 smithi043 ceph-mon[41444]: osdmap e235: 8 total, 8 up, 8 in 2024-02-05T07:39:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:30 smithi050 ceph-mon[33492]: pgmap v307: 292 pgs: 292 active+clean; 584 KiB data, 577 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:39:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-27"}]': finished 2024-02-05T07:39:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:30 smithi050 ceph-mon[33492]: osdmap e234: 8 total, 8 up, 8 in 2024-02-05T07:39:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81692-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:30 smithi050 ceph-mon[33492]: osdmap e235: 8 total, 8 up, 8 in 2024-02-05T07:39:31.286 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:39:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:39:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: pgmap v310: 276 pgs: 8 creating+peering, 268 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: osdmap e236: 8 total, 8 up, 8 in 2024-02-05T07:39:31.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:31.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:31.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:31.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: pgmap v310: 276 pgs: 8 creating+peering, 268 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: osdmap e236: 8 total, 8 up, 8 in 2024-02-05T07:39:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: pgmap v310: 276 pgs: 8 creating+peering, 268 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: osdmap e236: 8 total, 8 up, 8 in 2024-02-05T07:39:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[29879]: osdmap e237: 8 total, 8 up, 8 in 2024-02-05T07:39:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:33.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:33.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:33.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:33.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[41444]: osdmap e237: 8 total, 8 up, 8 in 2024-02-05T07:39:33.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:33.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:33.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:33 smithi050 ceph-mon[33492]: osdmap e237: 8 total, 8 up, 8 in 2024-02-05T07:39:33.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3168712808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]: dispatch 2024-02-05T07:39:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: pgmap v313: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: osdmap e238: 8 total, 8 up, 8 in 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: pgmap v313: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: osdmap e238: 8 total, 8 up, 8 in 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:34.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:34.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: pgmap v313: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:39:34.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-46"}]': finished 2024-02-05T07:39:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: osdmap e238: 8 total, 8 up, 8 in 2024-02-05T07:39:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]: dispatch 2024-02-05T07:39:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:34.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: osdmap e239: 8 total, 8 up, 8 in 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: osdmap e239: 8 total, 8 up, 8 in 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:35.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:35.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:35.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:35.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1131802510' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81692-37"}]': finished 2024-02-05T07:39:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81716-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: osdmap e239: 8 total, 8 up, 8 in 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: pgmap v316: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: osdmap e240: 8 total, 8 up, 8 in 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]': finished 2024-02-05T07:39:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]: dispatch 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: osdmap e241: 8 total, 8 up, 8 in 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]: dispatch 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: pgmap v316: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: osdmap e240: 8 total, 8 up, 8 in 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]': finished 2024-02-05T07:39:36.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]: dispatch 2024-02-05T07:39:36.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: osdmap e241: 8 total, 8 up, 8 in 2024-02-05T07:39:36.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]: dispatch 2024-02-05T07:39:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: pgmap v316: 292 pgs: 32 unknown, 260 active+clean; 584 KiB data, 586 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81692-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: osdmap e240: 8 total, 8 up, 8 in 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81716-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-29"}]': finished 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]: dispatch 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: osdmap e241: 8 total, 8 up, 8 in 2024-02-05T07:39:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]: dispatch 2024-02-05T07:39:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[41444]: pgmap v319: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]': finished 2024-02-05T07:39:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[41444]: osdmap e242: 8 total, 8 up, 8 in 2024-02-05T07:39:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[29879]: pgmap v319: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:37.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:37.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]': finished 2024-02-05T07:39:37.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:37 smithi043 ceph-mon[29879]: osdmap e242: 8 total, 8 up, 8 in 2024-02-05T07:39:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:37 smithi050 ceph-mon[33492]: pgmap v319: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:37 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81692-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-29", "mode": "writeback"}]': finished 2024-02-05T07:39:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:37 smithi050 ceph-mon[33492]: osdmap e242: 8 total, 8 up, 8 in 2024-02-05T07:39:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:38 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:38 smithi043 ceph-mon[41444]: osdmap e243: 8 total, 8 up, 8 in 2024-02-05T07:39:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:38 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:38 smithi043 ceph-mon[29879]: osdmap e243: 8 total, 8 up, 8 in 2024-02-05T07:39:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:38 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:38 smithi050 ceph-mon[33492]: osdmap e243: 8 total, 8 up, 8 in 2024-02-05T07:39:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:39:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:39:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:39:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: pgmap v322: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: osdmap e244: 8 total, 8 up, 8 in 2024-02-05T07:39:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: pgmap v322: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: osdmap e244: 8 total, 8 up, 8 in 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:40.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:40.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]: dispatch 2024-02-05T07:39:40.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:40 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]: dispatch 2024-02-05T07:39:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: pgmap v322: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: osdmap e244: 8 total, 8 up, 8 in 2024-02-05T07:39:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]: dispatch 2024-02-05T07:39:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]: dispatch 2024-02-05T07:39:40.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]: dispatch 2024-02-05T07:39:41.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:41.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:41.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]': finished 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: osdmap e245: 8 total, 8 up, 8 in 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81716-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]': finished 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81716-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: osdmap e246: 8 total, 8 up, 8 in 2024-02-05T07:39:41.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81716-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]': finished 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: osdmap e245: 8 total, 8 up, 8 in 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81716-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]': finished 2024-02-05T07:39:41.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:41.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81716-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:41.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: osdmap e246: 8 total, 8 up, 8 in 2024-02-05T07:39:41.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81716-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:39:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:39:41.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1580182628' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81716-47"}]': finished 2024-02-05T07:39:41.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:41.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.6", "id": [6, 7]}]': finished 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: osdmap e245: 8 total, 8 up, 8 in 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]: dispatch 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3390649934' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]: dispatch 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81716-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-29"}]': finished 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81692-38"}]': finished 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81716-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:41.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: osdmap e246: 8 total, 8 up, 8 in 2024-02-05T07:39:41.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81716-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[29879]: pgmap v325: 292 pgs: 1 peering, 291 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:39:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81692-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81692-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[41444]: pgmap v325: 292 pgs: 1 peering, 291 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:39:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81692-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81692-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:42 smithi050 ceph-mon[33492]: pgmap v325: 292 pgs: 1 peering, 291 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-02-05T07:39:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81692-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81692-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[29879]: osdmap e247: 8 total, 8 up, 8 in 2024-02-05T07:39:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81692-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[29879]: pgmap v328: 260 pgs: 1 peering, 259 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81716-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:43.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[29879]: osdmap e248: 8 total, 8 up, 8 in 2024-02-05T07:39:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[41444]: osdmap e247: 8 total, 8 up, 8 in 2024-02-05T07:39:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81692-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:43.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:43.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[41444]: pgmap v328: 260 pgs: 1 peering, 259 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:43.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81716-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:43.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:43 smithi043 ceph-mon[41444]: osdmap e248: 8 total, 8 up, 8 in 2024-02-05T07:39:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:43 smithi050 ceph-mon[33492]: osdmap e247: 8 total, 8 up, 8 in 2024-02-05T07:39:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81692-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:39:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:43 smithi050 ceph-mon[33492]: pgmap v328: 260 pgs: 1 peering, 259 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-02-05T07:39:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81716-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:43 smithi050 ceph-mon[33492]: osdmap e248: 8 total, 8 up, 8 in 2024-02-05T07:39:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81692-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[29879]: osdmap e249: 8 total, 8 up, 8 in 2024-02-05T07:39:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81692-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[41444]: osdmap e249: 8 total, 8 up, 8 in 2024-02-05T07:39:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81692-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:44 smithi050 ceph-mon[33492]: osdmap e249: 8 total, 8 up, 8 in 2024-02-05T07:39:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: pgmap v331: 308 pgs: 48 unknown, 1 peering, 259 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: osdmap e250: 8 total, 8 up, 8 in 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]': finished 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: osdmap e251: 8 total, 8 up, 8 in 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]: dispatch 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]: dispatch 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: pgmap v331: 308 pgs: 48 unknown, 1 peering, 259 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: osdmap e250: 8 total, 8 up, 8 in 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]': finished 2024-02-05T07:39:46.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: osdmap e251: 8 total, 8 up, 8 in 2024-02-05T07:39:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]: dispatch 2024-02-05T07:39:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:46.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]: dispatch 2024-02-05T07:39:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: pgmap v331: 308 pgs: 48 unknown, 1 peering, 259 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: osdmap e250: 8 total, 8 up, 8 in 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-31"}]': finished 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: osdmap e251: 8 total, 8 up, 8 in 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]: dispatch 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]: dispatch 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:46.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]: dispatch 2024-02-05T07:39:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]': finished 2024-02-05T07:39:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[41444]: osdmap e252: 8 total, 8 up, 8 in 2024-02-05T07:39:47.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:47.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:47.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:47.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:47.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]': finished 2024-02-05T07:39:47.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[29879]: osdmap e252: 8 total, 8 up, 8 in 2024-02-05T07:39:47.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:47 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1300093370' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81716-48"}]': finished 2024-02-05T07:39:47.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-31", "mode": "writeback"}]': finished 2024-02-05T07:39:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:47 smithi050 ceph-mon[33492]: osdmap e252: 8 total, 8 up, 8 in 2024-02-05T07:39:47.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]: dispatch 2024-02-05T07:39:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: pgmap v334: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-05T07:39:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: osdmap e253: 8 total, 8 up, 8 in 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:48.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: pgmap v334: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: osdmap e253: 8 total, 8 up, 8 in 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:48.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: pgmap v334: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-05T07:39:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3684483443' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81692-39"}]': finished 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81716-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: osdmap e253: 8 total, 8 up, 8 in 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]: dispatch 2024-02-05T07:39:48.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:39:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:39:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]': finished 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:49.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:49.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:49.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]': finished 2024-02-05T07:39:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-31"}]': finished 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[41444]: pgmap v337: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[41444]: osdmap e254: 8 total, 8 up, 8 in 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[41444]: osdmap e255: 8 total, 8 up, 8 in 2024-02-05T07:39:50.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[29879]: pgmap v337: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-05T07:39:50.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:50.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:50.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[29879]: osdmap e254: 8 total, 8 up, 8 in 2024-02-05T07:39:50.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:50.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:50.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:50 smithi043 ceph-mon[29879]: osdmap e255: 8 total, 8 up, 8 in 2024-02-05T07:39:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-mon[33492]: pgmap v337: 292 pgs: 292 active+clean; 4.6 MiB data, 608 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 2 B/s, 1 objects/s recovering 2024-02-05T07:39:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81692-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-mon[33492]: osdmap e254: 8 total, 8 up, 8 in 2024-02-05T07:39:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81716-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-mon[33492]: osdmap e255: 8 total, 8 up, 8 in 2024-02-05T07:39:51.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:39:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[41444]: pgmap v340: 268 pgs: 8 creating+peering, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[41444]: osdmap e256: 8 total, 8 up, 8 in 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:51.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[29879]: pgmap v340: 268 pgs: 8 creating+peering, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:51.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:51.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:51.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:51.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[29879]: osdmap e256: 8 total, 8 up, 8 in 2024-02-05T07:39:51.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:51.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:51.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:51 smithi050 ceph-mon[33492]: pgmap v340: 268 pgs: 8 creating+peering, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:51.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:51 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:39:51.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:51 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:39:51.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81692-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:51.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:51 smithi050 ceph-mon[33492]: osdmap e256: 8 total, 8 up, 8 in 2024-02-05T07:39:51.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:51.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:39:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[41444]: osdmap e257: 8 total, 8 up, 8 in 2024-02-05T07:39:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:53.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[29879]: osdmap e257: 8 total, 8 up, 8 in 2024-02-05T07:39:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:53.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:39:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:53 smithi050 ceph-mon[33492]: osdmap e257: 8 total, 8 up, 8 in 2024-02-05T07:39:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:53.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:39:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: pgmap v343: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: osdmap e258: 8 total, 8 up, 8 in 2024-02-05T07:39:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:54.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:54.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: pgmap v343: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: osdmap e258: 8 total, 8 up, 8 in 2024-02-05T07:39:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:54.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:54.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:54.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:54.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: pgmap v343: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:39:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:39:54.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: osdmap e258: 8 total, 8 up, 8 in 2024-02-05T07:39:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/487018641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]: dispatch 2024-02-05T07:39:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:54.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]': finished 2024-02-05T07:39:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: osdmap e259: 8 total, 8 up, 8 in 2024-02-05T07:39:55.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]': finished 2024-02-05T07:39:55.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]': finished 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: osdmap e259: 8 total, 8 up, 8 in 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]: dispatch 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]: dispatch 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]': finished 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:55.316 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-33"}]': finished 2024-02-05T07:39:55.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81716-49"}]': finished 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: osdmap e259: 8 total, 8 up, 8 in 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/462254178' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:55.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:39:55.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-33", "mode": "writeback"}]': finished 2024-02-05T07:39:55.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81692-40"}]': finished 2024-02-05T07:39:55.324 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81716-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: pgmap v346: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: osdmap e260: 8 total, 8 up, 8 in 2024-02-05T07:39:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81692-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81692-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: pgmap v346: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: osdmap e260: 8 total, 8 up, 8 in 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:56.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81692-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:56.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:56.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:56.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81692-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: pgmap v346: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 609 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:39:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: osdmap e260: 8 total, 8 up, 8 in 2024-02-05T07:39:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:39:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81692-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:39:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:39:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81692-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[29879]: osdmap e261: 8 total, 8 up, 8 in 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81692-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:39:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[41444]: osdmap e261: 8 total, 8 up, 8 in 2024-02-05T07:39:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81692-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:39:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:39:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:57 smithi050 ceph-mon[33492]: osdmap e261: 8 total, 8 up, 8 in 2024-02-05T07:39:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81692-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:39:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]: dispatch 2024-02-05T07:39:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:57 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:39:57.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81716-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[41444]: pgmap v349: 292 pgs: 292 active+clean; 4.6 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]': finished 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[41444]: osdmap e262: 8 total, 8 up, 8 in 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81692-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[41444]: osdmap e263: 8 total, 8 up, 8 in 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[29879]: pgmap v349: 292 pgs: 292 active+clean; 4.6 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:39:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]': finished 2024-02-05T07:39:58.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[29879]: osdmap e262: 8 total, 8 up, 8 in 2024-02-05T07:39:58.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:58.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81692-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:39:58.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:39:58 smithi043 ceph-mon[29879]: osdmap e263: 8 total, 8 up, 8 in 2024-02-05T07:39:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:58 smithi050 ceph-mon[33492]: pgmap v349: 292 pgs: 292 active+clean; 4.6 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:39:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-33"}]': finished 2024-02-05T07:39:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:58 smithi050 ceph-mon[33492]: osdmap e262: 8 total, 8 up, 8 in 2024-02-05T07:39:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:39:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81692-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:39:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:39:58 smithi050 ceph-mon[33492]: osdmap e263: 8 total, 8 up, 8 in 2024-02-05T07:39:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:39:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:39:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:40:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: pgmap v352: 276 pgs: 16 unknown, 260 active+clean; 4.6 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:40:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: osdmap e264: 8 total, 8 up, 8 in 2024-02-05T07:40:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:00.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: application not enabled on pool 'RoundTripPP2_smithi043-81716-50' 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: application not enabled on pool 'ExecuteClass_smithi043-81692-41' 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[29879]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: pgmap v352: 276 pgs: 16 unknown, 260 active+clean; 4.6 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: osdmap e264: 8 total, 8 up, 8 in 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:00.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: application not enabled on pool 'RoundTripPP2_smithi043-81716-50' 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: application not enabled on pool 'ExecuteClass_smithi043-81692-41' 2024-02-05T07:40:00.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:00 smithi043 ceph-mon[41444]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T07:40:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: pgmap v352: 276 pgs: 16 unknown, 260 active+clean; 4.6 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:40:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: osdmap e264: 8 total, 8 up, 8 in 2024-02-05T07:40:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: application not enabled on pool 'RoundTripPP2_smithi043-81716-50' 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: application not enabled on pool 'ExecuteClass_smithi043-81692-41' 2024-02-05T07:40:00.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-mon[33492]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T07:40:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:40:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: osdmap e265: 8 total, 8 up, 8 in 2024-02-05T07:40:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:40:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:01.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: osdmap e266: 8 total, 8 up, 8 in 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: osdmap e265: 8 total, 8 up, 8 in 2024-02-05T07:40:01.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: osdmap e266: 8 total, 8 up, 8 in 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:40:01.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:01.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:40:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:40:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: osdmap e265: 8 total, 8 up, 8 in 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/162316584' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]: dispatch 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81716-50"}]': finished 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: osdmap e266: 8 total, 8 up, 8 in 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]: dispatch 2024-02-05T07:40:01.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: pgmap v355: 292 pgs: 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: osdmap e267: 8 total, 8 up, 8 in 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]: dispatch 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]: dispatch 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3965199735' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: pgmap v355: 292 pgs: 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:40:02.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:02.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:40:02.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:02.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: osdmap e267: 8 total, 8 up, 8 in 2024-02-05T07:40:02.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]: dispatch 2024-02-05T07:40:02.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]: dispatch 2024-02-05T07:40:02.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3965199735' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: pgmap v355: 292 pgs: 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:40:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2737706500' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81692-41"}]': finished 2024-02-05T07:40:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: osdmap e267: 8 total, 8 up, 8 in 2024-02-05T07:40:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]: dispatch 2024-02-05T07:40:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]: dispatch 2024-02-05T07:40:02.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3965199735' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81692-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]': finished 2024-02-05T07:40:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3965199735' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:03.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81692-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: osdmap e268: 8 total, 8 up, 8 in 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81692-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81692-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]': finished 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3965199735' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81692-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: osdmap e268: 8 total, 8 up, 8 in 2024-02-05T07:40:03.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81692-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:03.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81692-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:03.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:03.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-35", "mode": "writeback"}]': finished 2024-02-05T07:40:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3965199735' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81716-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81692-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: osdmap e268: 8 total, 8 up, 8 in 2024-02-05T07:40:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81692-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:04 smithi043 ceph-mon[29879]: pgmap v358: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:04 smithi043 ceph-mon[41444]: pgmap v358: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:04 smithi050 ceph-mon[33492]: pgmap v358: 324 pgs: 32 unknown, 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: osdmap e269: 8 total, 8 up, 8 in 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: pgmap v361: 292 pgs: 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81692-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: osdmap e270: 8 total, 8 up, 8 in 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81716-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: osdmap e269: 8 total, 8 up, 8 in 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: pgmap v361: 292 pgs: 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81692-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: osdmap e270: 8 total, 8 up, 8 in 2024-02-05T07:40:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81716-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: osdmap e269: 8 total, 8 up, 8 in 2024-02-05T07:40:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: pgmap v361: 292 pgs: 292 active+clean; 4.6 MiB data, 615 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81692-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: osdmap e270: 8 total, 8 up, 8 in 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81716-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[29879]: osdmap e271: 8 total, 8 up, 8 in 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[41444]: osdmap e271: 8 total, 8 up, 8 in 2024-02-05T07:40:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:06 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:06 smithi050 ceph-mon[33492]: osdmap e271: 8 total, 8 up, 8 in 2024-02-05T07:40:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[29879]: pgmap v364: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81716-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[29879]: osdmap e272: 8 total, 8 up, 8 in 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[41444]: pgmap v364: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81716-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:08.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:08.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[41444]: osdmap e272: 8 total, 8 up, 8 in 2024-02-05T07:40:08.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:08.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:08 smithi050 ceph-mon[33492]: pgmap v364: 300 pgs: 8 creating+peering, 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:40:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81716-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:08 smithi050 ceph-mon[33492]: osdmap e272: 8 total, 8 up, 8 in 2024-02-05T07:40:08.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:09.059 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: sta api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7029 ms) 2024-02-05T07:40:09.059 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-02-05T07:40:09.059 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6944 ms) 2024-02-05T07:40:09.059 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-05T07:40:09.059 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7014 ms) 2024-02-05T07:40:09.059 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-05T07:40:09.059 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6998 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7030 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7036 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7059 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7017 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7125 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (6758 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7018 ms) 2024-02-05T07:40:09.060 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6971 ms) 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6999 ms) 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7017 ms) 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7019 ms) 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7017 ms) 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112052 ms total) 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment tear-down 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (205400 ms total) 2024-02-05T07:40:09.061 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ PASSED ] 42 tests. 2024-02-05T07:40:09.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:40:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:40:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[29879]: osdmap e273: 8 total, 8 up, 8 in 2024-02-05T07:40:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:09.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:09.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:09.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[41444]: osdmap e273: 8 total, 8 up, 8 in 2024-02-05T07:40:09.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:09.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:09.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:09 smithi050 ceph-mon[33492]: osdmap e273: 8 total, 8 up, 8 in 2024-02-05T07:40:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]: dispatch 2024-02-05T07:40:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:09.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[41444]: pgmap v367: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:40:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[41444]: osdmap e274: 8 total, 8 up, 8 in 2024-02-05T07:40:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:10.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[29879]: pgmap v367: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:40:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[29879]: osdmap e274: 8 total, 8 up, 8 in 2024-02-05T07:40:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:10.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-mon[33492]: pgmap v367: 300 pgs: 8 unknown, 292 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:40:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/774675286' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81692-42"}]': finished 2024-02-05T07:40:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-mon[33492]: osdmap e274: 8 total, 8 up, 8 in 2024-02-05T07:40:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]: dispatch 2024-02-05T07:40:10.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:11.313 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:40:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:40:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: osdmap e275: 8 total, 8 up, 8 in 2024-02-05T07:40:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:11 smithi050 ceph-mon[33492]: osdmap e276: 8 total, 8 up, 8 in 2024-02-05T07:40:11.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: osdmap e275: 8 total, 8 up, 8 in 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[29879]: osdmap e276: 8 total, 8 up, 8 in 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-35"}]': finished 2024-02-05T07:40:11.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: osdmap e275: 8 total, 8 up, 8 in 2024-02-05T07:40:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]: dispatch 2024-02-05T07:40:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1449670268' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81716-52"}]': finished 2024-02-05T07:40:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:11 smithi043 ceph-mon[41444]: osdmap e276: 8 total, 8 up, 8 in 2024-02-05T07:40:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: pgmap v370: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:40:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:12.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:12.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:12.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: osdmap e277: 8 total, 8 up, 8 in 2024-02-05T07:40:12.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:12.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:12.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: pgmap v370: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:40:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: osdmap e277: 8 total, 8 up, 8 in 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: pgmap v370: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:12.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: osdmap e277: 8 total, 8 up, 8 in 2024-02-05T07:40:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:12.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[29879]: pgmap v373: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[29879]: osdmap e278: 8 total, 8 up, 8 in 2024-02-05T07:40:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[41444]: pgmap v373: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[41444]: osdmap e278: 8 total, 8 up, 8 in 2024-02-05T07:40:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:13 smithi050 ceph-mon[33492]: pgmap v373: 292 pgs: 32 unknown, 260 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:13 smithi050 ceph-mon[33492]: osdmap e278: 8 total, 8 up, 8 in 2024-02-05T07:40:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[29879]: osdmap e279: 8 total, 8 up, 8 in 2024-02-05T07:40:15.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:15.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[41444]: osdmap e279: 8 total, 8 up, 8 in 2024-02-05T07:40:15.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:15.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:15.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81716-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:15.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:15 smithi050 ceph-mon[33492]: osdmap e279: 8 total, 8 up, 8 in 2024-02-05T07:40:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:15.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: pgmap v376: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]': finished 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: osdmap e280: 8 total, 8 up, 8 in 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]: dispatch 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]: dispatch 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]': finished 2024-02-05T07:40:16.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: osdmap e281: 8 total, 8 up, 8 in 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: pgmap v376: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]': finished 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: osdmap e280: 8 total, 8 up, 8 in 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]: dispatch 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]: dispatch 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]': finished 2024-02-05T07:40:16.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: osdmap e281: 8 total, 8 up, 8 in 2024-02-05T07:40:16.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:16.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:16.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:16.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: pgmap v376: 300 pgs: 40 unknown, 260 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-37"}]': finished 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: osdmap e280: 8 total, 8 up, 8 in 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]: dispatch 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]: dispatch 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-37", "mode": "writeback"}]': finished 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: osdmap e281: 8 total, 8 up, 8 in 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: pgmap v379: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: osdmap e282: 8 total, 8 up, 8 in 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:18.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: pgmap v379: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: osdmap e282: 8 total, 8 up, 8 in 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:18.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:18 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:18.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: pgmap v379: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:18.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:18.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:18.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: osdmap e282: 8 total, 8 up, 8 in 2024-02-05T07:40:18.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/558079604' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]: dispatch 2024-02-05T07:40:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]: dispatch 2024-02-05T07:40:18.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:18 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:19.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:40:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:40:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]': finished 2024-02-05T07:40:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: osdmap e283: 8 total, 8 up, 8 in 2024-02-05T07:40:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:40:19.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]': finished 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: osdmap e283: 8 total, 8 up, 8 in 2024-02-05T07:40:19.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:19.315 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:19 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81716-53"}]': finished 2024-02-05T07:40:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-37"}]': finished 2024-02-05T07:40:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: osdmap e283: 8 total, 8 up, 8 in 2024-02-05T07:40:19.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:19.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:19 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: pgmap v382: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: osdmap e284: 8 total, 8 up, 8 in 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:40:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: pgmap v382: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: osdmap e284: 8 total, 8 up, 8 in 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:40:20.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:20 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: pgmap v382: 292 pgs: 292 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81716-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: osdmap e284: 8 total, 8 up, 8 in 2024-02-05T07:40:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:40:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:40:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:40:21.282 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:40:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:40:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: osdmap e285: 8 total, 8 up, 8 in 2024-02-05T07:40:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: pgmap v385: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:40:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: osdmap e286: 8 total, 8 up, 8 in 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: osdmap e285: 8 total, 8 up, 8 in 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: pgmap v385: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: osdmap e286: 8 total, 8 up, 8 in 2024-02-05T07:40:21.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:21.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: osdmap e285: 8 total, 8 up, 8 in 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: pgmap v385: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81716-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: osdmap e286: 8 total, 8 up, 8 in 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[29879]: osdmap e287: 8 total, 8 up, 8 in 2024-02-05T07:40:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:23.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:23.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[41444]: osdmap e287: 8 total, 8 up, 8 in 2024-02-05T07:40:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:23.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:23 smithi050 ceph-mon[33492]: osdmap e287: 8 total, 8 up, 8 in 2024-02-05T07:40:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:23.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[41444]: pgmap v388: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:40:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]': finished 2024-02-05T07:40:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[41444]: osdmap e288: 8 total, 8 up, 8 in 2024-02-05T07:40:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]: dispatch 2024-02-05T07:40:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]: dispatch 2024-02-05T07:40:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:24.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:24.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[29879]: pgmap v388: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:40:24.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]': finished 2024-02-05T07:40:24.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[29879]: osdmap e288: 8 total, 8 up, 8 in 2024-02-05T07:40:24.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]: dispatch 2024-02-05T07:40:24.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]: dispatch 2024-02-05T07:40:24.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:24.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:24 smithi050 ceph-mon[33492]: pgmap v388: 300 pgs: 8 unknown, 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:40:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-39"}]': finished 2024-02-05T07:40:24.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:24 smithi050 ceph-mon[33492]: osdmap e288: 8 total, 8 up, 8 in 2024-02-05T07:40:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]: dispatch 2024-02-05T07:40:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]: dispatch 2024-02-05T07:40:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:24.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]': finished 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: osdmap e289: 8 total, 8 up, 8 in 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:25.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]': finished 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: osdmap e289: 8 total, 8 up, 8 in 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:25.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:40:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-39", "mode": "writeback"}]': finished 2024-02-05T07:40:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: osdmap e289: 8 total, 8 up, 8 in 2024-02-05T07:40:25.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3899903813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]: dispatch 2024-02-05T07:40:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:25.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: pgmap v391: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: osdmap e290: 8 total, 8 up, 8 in 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: pgmap v391: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: osdmap e290: 8 total, 8 up, 8 in 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:26.315 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: pgmap v391: 292 pgs: 32 creating+peering, 260 active+clean; 4.6 MiB data, 605 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81716-54"}]': finished 2024-02-05T07:40:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:40:26.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: osdmap e290: 8 total, 8 up, 8 in 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]: dispatch 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:26.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: seed 81993 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (408 ms) 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (14724 ms) 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (2 ms) 2024-02-05T07:40:27.293 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (15134 ms total) 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (9091 ms) 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8923 ms) 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10005 ms) 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [3] 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [4,3] 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [5,4,3] 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting some heads 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-05T07:40:27.294 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 3 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46089 ms) 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9942 ms) 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8001 ms) 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8035 ms) 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8305 ms) 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-05T07:40:27.295 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (9795 ms) 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8869 ms) 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10026 ms) 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12241 ms) 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8003 ms) 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-05T07:40:27.296 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (7766 ms) 2024-02-05T07:40:27.297 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-05T07:40:27.297 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13026 ms) 2024-02-05T07:40:27.297 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-05T07:40:27.297 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7963 ms) 2024-02-05T07:40:27.297 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-05T07:40:27.297 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8276 ms) 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]': finished 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[29879]: osdmap e291: 8 total, 8 up, 8 in 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]': finished 2024-02-05T07:40:27.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[41444]: osdmap e291: 8 total, 8 up, 8 in 2024-02-05T07:40:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:27.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:27 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:40:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-39"}]': finished 2024-02-05T07:40:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81716-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:27 smithi050 ceph-mon[33492]: osdmap e291: 8 total, 8 up, 8 in 2024-02-05T07:40:27.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:27.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:28.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:28 smithi043 ceph-mon[29879]: pgmap v394: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-02-05T07:40:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:28 smithi043 ceph-mon[29879]: osdmap e292: 8 total, 8 up, 8 in 2024-02-05T07:40:28.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:28.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:28 smithi043 ceph-mon[41444]: pgmap v394: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-02-05T07:40:28.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:28 smithi043 ceph-mon[41444]: osdmap e292: 8 total, 8 up, 8 in 2024-02-05T07:40:28.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:28.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:28 smithi050 ceph-mon[33492]: pgmap v394: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 76 op/s 2024-02-05T07:40:28.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:28 smithi050 ceph-mon[33492]: osdmap e292: 8 total, 8 up, 8 in 2024-02-05T07:40:28.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:29.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:40:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:40:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[29879]: osdmap e293: 8 total, 8 up, 8 in 2024-02-05T07:40:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:29.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[41444]: osdmap e293: 8 total, 8 up, 8 in 2024-02-05T07:40:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:29.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81716-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:29 smithi050 ceph-mon[33492]: osdmap e293: 8 total, 8 up, 8 in 2024-02-05T07:40:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:29.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[29879]: pgmap v397: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[29879]: osdmap e294: 8 total, 8 up, 8 in 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[41444]: pgmap v397: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[41444]: osdmap e294: 8 total, 8 up, 8 in 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:30.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:30 smithi050 ceph-mon[33492]: pgmap v397: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-05T07:40:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:30 smithi050 ceph-mon[33492]: osdmap e294: 8 total, 8 up, 8 in 2024-02-05T07:40:30.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:30.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:30.415 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRrted 0x55567c1a7510 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7ae0 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7080 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a6d10 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7190 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a72a0 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7640 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7750 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7860 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7970 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a80d0 2024-02-05T07:40:30.416 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7c30 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7d60 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7e70 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a7fc0 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a86a0 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8200 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8330 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8440 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8590 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8c90 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a87f0 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8940 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8a50 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8b80 2024-02-05T07:40:30.417 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a95e0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a9140 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a9270 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a93a0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a9730 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a9c20 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aa0b0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aa1e0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8da0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a8ed0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a9000 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a9880 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a94d0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a99b0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1a9ac0 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aa310 2024-02-05T07:40:30.418 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aa460 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aaa50 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aa5b0 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aa700 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aa830 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aab80 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ab010 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ab140 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aac90 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aade0 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ab710 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ab270 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ab840 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1abcd0 2024-02-05T07:40:30.419 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac160 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac290 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac860 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1abe00 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1abf10 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac040 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac9b0 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac3c0 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac510 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac620 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ac750 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1acf80 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1acae0 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1acbf0 2024-02-05T07:40:30.420 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1acd40 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ace70 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad590 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad0f0 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad200 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad330 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad480 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1adb60 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad6c0 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad7f0 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ad900 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ada30 2024-02-05T07:40:30.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae130 2024-02-05T07:40:31.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:40:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:40:31.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: osdmap e295: 8 total, 8 up, 8 in 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: osdmap e296: 8 total, 8 up, 8 in 2024-02-05T07:40:31.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: osdmap e295: 8 total, 8 up, 8 in 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: osdmap e296: 8 total, 8 up, 8 in 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: osdmap e295: 8 total, 8 up, 8 in 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:40:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: osdmap e296: 8 total, 8 up, 8 in 2024-02-05T07:40:31.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:32.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: pgmap v400: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:32.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: osdmap e297: 8 total, 8 up, 8 in 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:40:32.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: pgmap v400: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: osdmap e297: 8 total, 8 up, 8 in 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:40:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: pgmap v400: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/790079044' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]: dispatch 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81716-55"}]': finished 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: osdmap e297: 8 total, 8 up, 8 in 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:40:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: osdmap e298: 8 total, 8 up, 8 in 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: osdmap e298: 8 total, 8 up, 8 in 2024-02-05T07:40:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81716-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: osdmap e298: 8 total, 8 up, 8 in 2024-02-05T07:40:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:33.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:34 smithi050 ceph-mon[33492]: pgmap v403: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:34 smithi043 ceph-mon[29879]: pgmap v403: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:34 smithi043 ceph-mon[41444]: pgmap v403: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: osdmap e299: 8 total, 8 up, 8 in 2024-02-05T07:40:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]: dispatch 2024-02-05T07:40:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]: dispatch 2024-02-05T07:40:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: pgmap v406: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]': finished 2024-02-05T07:40:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:35 smithi050 ceph-mon[33492]: osdmap e300: 8 total, 8 up, 8 in 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: osdmap e299: 8 total, 8 up, 8 in 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]: dispatch 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]: dispatch 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: pgmap v406: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]': finished 2024-02-05T07:40:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[29879]: osdmap e300: 8 total, 8 up, 8 in 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: osdmap e299: 8 total, 8 up, 8 in 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]: dispatch 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]: dispatch 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: pgmap v406: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81716-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-41"}]': finished 2024-02-05T07:40:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:35 smithi043 ceph-mon[41444]: osdmap e300: 8 total, 8 up, 8 in 2024-02-05T07:40:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:36 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:36 smithi043 ceph-mon[29879]: osdmap e301: 8 total, 8 up, 8 in 2024-02-05T07:40:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:36 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:36 smithi043 ceph-mon[41444]: osdmap e301: 8 total, 8 up, 8 in 2024-02-05T07:40:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:36 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:36 smithi050 ceph-mon[33492]: osdmap e301: 8 total, 8 up, 8 in 2024-02-05T07:40:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[41444]: pgmap v409: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[41444]: osdmap e302: 8 total, 8 up, 8 in 2024-02-05T07:40:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:38.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[29879]: pgmap v409: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:38.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[29879]: osdmap e302: 8 total, 8 up, 8 in 2024-02-05T07:40:38.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:38.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:38.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:38.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:38 smithi050 ceph-mon[33492]: pgmap v409: 268 pgs: 8 creating+peering, 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:38 smithi050 ceph-mon[33492]: osdmap e302: 8 total, 8 up, 8 in 2024-02-05T07:40:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:40:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:40:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: osdmap e303: 8 total, 8 up, 8 in 2024-02-05T07:40:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81993-6","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:40:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: osdmap e304: 8 total, 8 up, 8 in 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: osdmap e303: 8 total, 8 up, 8 in 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81993-6","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: osdmap e304: 8 total, 8 up, 8 in 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:40:39.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:40:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: osdmap e303: 8 total, 8 up, 8 in 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81993-6","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3842137688' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]: dispatch 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81716-56"}]': finished 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: osdmap e304: 8 total, 8 up, 8 in 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:40:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]: dispatch 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: pgmap v412: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]': finished 2024-02-05T07:40:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: osdmap e305: 8 total, 8 up, 8 in 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: pgmap v412: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:40.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]': finished 2024-02-05T07:40:40.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:40.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: osdmap e305: 8 total, 8 up, 8 in 2024-02-05T07:40:40.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:40.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: pgmap v412: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_count","val": "8"}]': finished 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81716-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: osdmap e305: 8 total, 8 up, 8 in 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:40:41.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:40:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:40:42.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc:ead 2024-02-05T07:40:42.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-05T07:40:42.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 274:602f83fe:::foo:head 2024-02-05T07:40:42.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (8774 ms) 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 0 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 1 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 2 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 3 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 4 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 5 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 6 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 7 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 8 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 9 ls 1707118843,0 2024-02-05T07:40:42.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 10 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 11 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 12 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 13 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 14 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 15 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 16 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 17 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 18 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 19 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 20 ls 1707118843,0 2024-02-05T07:40:42.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 21 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 22 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 23 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 24 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 25 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 26 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 27 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 28 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 29 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 30 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 31 ls 1707118843,0 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6cac518f:::0:head 2024-02-05T07:40:42.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:02547ec2:::1:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f905c69b:::2:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cfc208b3:::3:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d83876eb:::4:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b29083e3:::5:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c4fdafeb:::6:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5c6b0b28:::7:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bd63b0f1:::8:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e960b815:::9:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:52ea6a34:::10:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:89d3ae78:::11:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:de5d7c5f:::12:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:566253c9:::13:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:62a1935d:::14:head 2024-02-05T07:40:42.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:863748b0:::15:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3958e169:::16:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4d4dabf9:::17:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:8391935d:::18:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:28883081:::19:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:69259c59:::20:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4bdb80b7:::21:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a11c5d71:::22:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:271af37b:::23:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:95b121be:::24:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:58d1031b:::25:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0a050783:::26:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c709704c:::27:head 2024-02-05T07:40:42.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cbe56eaf:::28:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:86b4b162:::29:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:70d89383:::30:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:dd450c7c:::31:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6d5729b1:::32:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c388f3fb:::33:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:56cfea31:::34:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9dbc1bf7:::35:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:40b74ccd:::36:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4d5aaf42:::37:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:920f362c:::38:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6cc53222:::39:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9cad833f:::40:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1ea84d41:::41:head 2024-02-05T07:40:42.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c4480ef6:::42:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a694361e:::43:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d1bd33e9:::44:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ddc2cd5d:::45:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:2b782207:::46:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7b187fca:::47:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:90ecdf6f:::48:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a5ed95fe:::49:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ea0eaa55:::50:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f33ef17b:::51:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a0d1b2f6:::52:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:60c5229e:::53:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:edcbc575:::54:head 2024-02-05T07:40:42.318 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:102cf253:::55:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:efb7fb0b:::56:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:50d0a326:::57:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d4dc5daf:::58:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3a130462:::59:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ec87ed71:::60:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d5bc9454:::61:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3ddfe313:::62:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7c2816b9:::63:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:47e00e4d:::64:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c6410c18:::65:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b48ed237:::66:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cd63ad31:::67:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b179e92b:::68:head 2024-02-05T07:40:42.319 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0d9f741a:::69:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6d3352ae:::70:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c6d5c19e:::71:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bc4729c3:::72:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:77e930b9:::73:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0abeecfd:::74:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b7c37e15:::75:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b6378398:::76:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:02bd68de:::77:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:cc795d2d:::78:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:630d4fea:::79:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e0d29ef5:::80:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:fd6f13d2:::81:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:606461d5:::82:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:eadbdc43:::83:head 2024-02-05T07:40:42.320 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:8761d0bb:::84:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9ef0186f:::85:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e0d41294:::86:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:961de695:::87:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1423148f:::88:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:633a8fa2:::89:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a8653809:::90:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3dac8b33:::91:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:35aad435:::92:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f6dcc343:::93:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:dbbdad87:::94:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1cb48ce0:::95:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:03cd461c:::96:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:17a4ea99:::97:head 2024-02-05T07:40:42.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9993c9a7:::98:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6394211c:::99:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:94c7ae57:::100:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6fdee5bb:::101:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9a477fd1:::102:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:eb850916:::103:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:affc56b9:::104:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b42dc814:::105:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f319f8f0:::106:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9a40b9de:::107:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:8b524f28:::108:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e3de589f:::109:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:90f90a5b:::110:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a7b4f1d7:::111:head 2024-02-05T07:40:42.322 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:af51766e:::112:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b6f90bd1:::113:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e0261208:::114:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c9569ef7:::115:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:61bebe50:::116:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:fe93412b:::117:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d3d38bee:::118:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3100ba0c:::119:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d0560ada:::120:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f0ea8b35:::121:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:766f231a:::122:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a07a2582:::123:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bd7c6b3a:::124:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:fb2ddaff:::125:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4408e1fe:::126:head 2024-02-05T07:40:42.323 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ee1df7a7:::127:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c3002909:::128:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4f48ffa9:::129:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:edf38733:::130:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c08425c0:::131:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5f902d98:::132:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:41ea2c93:::133:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:813cee13:::134:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0131818d:::135:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:26ba5a85:::136:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:381b8a5a:::137:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:28797e47:::138:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:bfca7f22:::139:head 2024-02-05T07:40:42.324 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:36807075:::140:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:80b03975:::141:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5c15709b:::142:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f39ea15e:::143:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ea992956:::144:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:48887b1c:::145:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:9f24a9dd:::146:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:987f100b:::147:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d2dd3581:::148:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7fed1808:::149:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c80b70e9:::150:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:85ed90f9:::151:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:36428b24:::152:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d044c34a:::153:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7c18bf58:::154:head 2024-02-05T07:40:42.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d1c21232:::155:head 2024-02-05T07:40:42.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a7a3c575:::156:head 2024-02-05T07:40:42.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:87da0633:::157:head 2024-02-05T07:40:42.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d5ac3822:::158:head 2024-02-05T07:40:42.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3f20522d:::159:head 2024-02-05T07:40:42.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6ca26563:::160:head 2024-02-05T07:40:42.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:532ce135:::161:head 2024-02-05T07:40:42.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:c78863e6:::162:head 2024-02-05T07:40:42.326 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:42 smithi050 ceph-mon[33492]: pgmap v415: 292 pgs: 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:42.326 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:42.326 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:40:42.327 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:40:42.327 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:42 smithi050 ceph-mon[33492]: osdmap e306: 8 total, 8 up, 8 in 2024-02-05T07:40:42.327 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:40:42.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[29879]: pgmap v415: 292 pgs: 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[29879]: osdmap e306: 8 total, 8 up, 8 in 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[41444]: pgmap v415: 292 pgs: 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[41444]: osdmap e306: 8 total, 8 up, 8 in 2024-02-05T07:40:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: osdmap e307: 8 total, 8 up, 8 in 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81993-43","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]: dispatch 2024-02-05T07:40:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]: dispatch 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: osdmap e307: 8 total, 8 up, 8 in 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81993-43","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]: dispatch 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]: dispatch 2024-02-05T07:40:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81716-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-05T07:40:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: osdmap e307: 8 total, 8 up, 8 in 2024-02-05T07:40:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81993-43","var": "pg_num","format": "json"}]: dispatch 2024-02-05T07:40:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:40:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]: dispatch 2024-02-05T07:40:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]: dispatch 2024-02-05T07:40:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[29879]: pgmap v418: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]': finished 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[29879]: osdmap e308: 8 total, 8 up, 8 in 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[41444]: pgmap v418: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]': finished 2024-02-05T07:40:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:44 smithi043 ceph-mon[41444]: osdmap e308: 8 total, 8 up, 8 in 2024-02-05T07:40:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:44 smithi050 ceph-mon[33492]: pgmap v418: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:44 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-43"}]': finished 2024-02-05T07:40:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:44 smithi050 ceph-mon[33492]: osdmap e308: 8 total, 8 up, 8 in 2024-02-05T07:40:45.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:45 smithi043 ceph-mon[29879]: osdmap e309: 8 total, 8 up, 8 in 2024-02-05T07:40:45.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:45.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:45 smithi043 ceph-mon[41444]: osdmap e309: 8 total, 8 up, 8 in 2024-02-05T07:40:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:45 smithi050 ceph-mon[33492]: osdmap e309: 8 total, 8 up, 8 in 2024-02-05T07:40:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[29879]: pgmap v421: 260 pgs: 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[29879]: osdmap e310: 8 total, 8 up, 8 in 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[41444]: pgmap v421: 260 pgs: 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:46.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[41444]: osdmap e310: 8 total, 8 up, 8 in 2024-02-05T07:40:46.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:46.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:46.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:46.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:46 smithi050 ceph-mon[33492]: pgmap v421: 260 pgs: 260 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:46 smithi050 ceph-mon[33492]: osdmap e310: 8 total, 8 up, 8 in 2024-02-05T07:40:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3407715055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:46.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]: dispatch 2024-02-05T07:40:46.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:46.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: osdmap e311: 8 total, 8 up, 8 in 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81716-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: osdmap e311: 8 total, 8 up, 8 in 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81716-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81716-57"}]': finished 2024-02-05T07:40:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:40:47.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: osdmap e311: 8 total, 8 up, 8 in 2024-02-05T07:40:47.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:47.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:40:47.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:47.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:47.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81716-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:48.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[29879]: pgmap v424: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81716-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[29879]: osdmap e312: 8 total, 8 up, 8 in 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81716-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[41444]: pgmap v424: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81716-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:40:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[41444]: osdmap e312: 8 total, 8 up, 8 in 2024-02-05T07:40:48.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81716-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:48.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:40:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:48 smithi050 ceph-mon[33492]: pgmap v424: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:40:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81716-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:40:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:48 smithi050 ceph-mon[33492]: osdmap e312: 8 total, 8 up, 8 in 2024-02-05T07:40:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81716-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:48.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:40:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:40:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:40:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:40:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:40:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[29879]: osdmap e313: 8 total, 8 up, 8 in 2024-02-05T07:40:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:40:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:40:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:40:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[41444]: osdmap e313: 8 total, 8 up, 8 in 2024-02-05T07:40:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:40:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:40:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:40:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:49 smithi050 ceph-mon[33492]: osdmap e313: 8 total, 8 up, 8 in 2024-02-05T07:40:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[41444]: pgmap v427: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81716-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[41444]: osdmap e314: 8 total, 8 up, 8 in 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[29879]: pgmap v427: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81716-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:40:50.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[29879]: osdmap e314: 8 total, 8 up, 8 in 2024-02-05T07:40:50.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:40:50.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:40:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:50 smithi050 ceph-mon[33492]: pgmap v427: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81716-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:40:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:50 smithi050 ceph-mon[33492]: osdmap e314: 8 total, 8 up, 8 in 2024-02-05T07:40:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:40:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:40:51.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:40:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: osdmap e315: 8 total, 8 up, 8 in 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: osdmap e316: 8 total, 8 up, 8 in 2024-02-05T07:40:51.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: osdmap e315: 8 total, 8 up, 8 in 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: osdmap e316: 8 total, 8 up, 8 in 2024-02-05T07:40:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: osdmap e315: 8 total, 8 up, 8 in 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: osdmap e316: 8 total, 8 up, 8 in 2024-02-05T07:40:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[29879]: pgmap v430: 300 pgs: 300 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:40:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[29879]: osdmap e317: 8 total, 8 up, 8 in 2024-02-05T07:40:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[41444]: pgmap v430: 300 pgs: 300 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:40:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[41444]: osdmap e317: 8 total, 8 up, 8 in 2024-02-05T07:40:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:52 smithi050 ceph-mon[33492]: pgmap v430: 300 pgs: 300 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:40:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:40:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:52 smithi050 ceph-mon[33492]: osdmap e317: 8 total, 8 up, 8 in 2024-02-05T07:40:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]: dispatch 2024-02-05T07:40:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[29879]: pgmap v433: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[29879]: osdmap e318: 8 total, 8 up, 8 in 2024-02-05T07:40:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:54.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[41444]: pgmap v433: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[41444]: osdmap e318: 8 total, 8 up, 8 in 2024-02-05T07:40:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:54 smithi050 ceph-mon[33492]: pgmap v433: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:40:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/851253622' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81716-58"}]': finished 2024-02-05T07:40:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:54 smithi050 ceph-mon[33492]: osdmap e318: 8 total, 8 up, 8 in 2024-02-05T07:40:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:40:55.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:55.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:55 smithi050 ceph-mon[33492]: osdmap e319: 8 total, 8 up, 8 in 2024-02-05T07:40:55.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81716-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:55 smithi043 ceph-mon[41444]: osdmap e319: 8 total, 8 up, 8 in 2024-02-05T07:40:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81716-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:40:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:55 smithi043 ceph-mon[29879]: osdmap e319: 8 total, 8 up, 8 in 2024-02-05T07:40:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81716-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:56.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:56 smithi043 ceph-mon[29879]: pgmap v436: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:56 smithi043 ceph-mon[29879]: osdmap e320: 8 total, 8 up, 8 in 2024-02-05T07:40:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:56 smithi043 ceph-mon[41444]: pgmap v436: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:56 smithi043 ceph-mon[41444]: osdmap e320: 8 total, 8 up, 8 in 2024-02-05T07:40:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:56 smithi050 ceph-mon[33492]: pgmap v436: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:40:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:56 smithi050 ceph-mon[33492]: osdmap e320: 8 total, 8 up, 8 in 2024-02-05T07:40:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81716-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:40:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:57 smithi043 ceph-mon[29879]: osdmap e321: 8 total, 8 up, 8 in 2024-02-05T07:40:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81716-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:40:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:57 smithi043 ceph-mon[41444]: osdmap e321: 8 total, 8 up, 8 in 2024-02-05T07:40:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81716-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:40:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:57 smithi050 ceph-mon[33492]: osdmap e321: 8 total, 8 up, 8 in 2024-02-05T07:40:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[29879]: pgmap v439: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:40:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[29879]: osdmap e322: 8 total, 8 up, 8 in 2024-02-05T07:40:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[41444]: pgmap v439: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:40:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[41444]: osdmap e322: 8 total, 8 up, 8 in 2024-02-05T07:40:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:58 smithi050 ceph-mon[33492]: pgmap v439: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:40:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:58 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:40:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:58 smithi050 ceph-mon[33492]: osdmap e322: 8 total, 8 up, 8 in 2024-02-05T07:40:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:40:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:40:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:40:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[29879]: osdmap e323: 8 total, 8 up, 8 in 2024-02-05T07:40:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[29879]: pgmap v442: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:40:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:40:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[29879]: osdmap e324: 8 total, 8 up, 8 in 2024-02-05T07:40:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[41444]: osdmap e323: 8 total, 8 up, 8 in 2024-02-05T07:40:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[41444]: pgmap v442: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:40:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:40:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:40:59 smithi043 ceph-mon[41444]: osdmap e324: 8 total, 8 up, 8 in 2024-02-05T07:40:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:59 smithi050 ceph-mon[33492]: osdmap e323: 8 total, 8 up, 8 in 2024-02-05T07:40:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:40:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:59 smithi050 ceph-mon[33492]: pgmap v442: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:40:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:40:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:40:59 smithi050 ceph-mon[33492]: osdmap e324: 8 total, 8 up, 8 in 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7462ddf6:::16o.RoundTripAppendPP (3007 ms) 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3255 ms) 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2737 ms) 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3010 ms) 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: setting pool EIO 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: max_success 99, min_failed 100 2024-02-05T07:41:00.174 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4019 ms) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3020 ms) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (129372 ms total) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3012 ms) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9053 ms) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14897 ms) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3013 ms) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29975 ms total) 2024-02-05T07:41:00.175 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3005 ms) 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3005 ms total) 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13989 ms) 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6997 ms) 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7019 ms) 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7013 ms) 2024-02-05T07:41:00.176 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3260 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (6766 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6962 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7025 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7019 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7013 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7029 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7032 ms) 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-05T07:41:00.177 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7038 ms) 2024-02-05T07:41:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:41:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:41:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:00 smithi043 ceph-mon[29879]: osdmap e325: 8 total, 8 up, 8 in 2024-02-05T07:41:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:41:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:41:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:00 smithi043 ceph-mon[41444]: osdmap e325: 8 total, 8 up, 8 in 2024-02-05T07:41:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]: dispatch 2024-02-05T07:41:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/253842600' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81716-59"}]': finished 2024-02-05T07:41:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:00 smithi050 ceph-mon[33492]: osdmap e325: 8 total, 8 up, 8 in 2024-02-05T07:41:01.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:41:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[29879]: pgmap v445: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[29879]: osdmap e326: 8 total, 8 up, 8 in 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4149899887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[41444]: pgmap v445: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[41444]: osdmap e326: 8 total, 8 up, 8 in 2024-02-05T07:41:02.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4149899887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:02 smithi050 ceph-mon[33492]: pgmap v445: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:41:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:02 smithi050 ceph-mon[33492]: osdmap e326: 8 total, 8 up, 8 in 2024-02-05T07:41:02.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4149899887' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4149899887' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:03 smithi043 ceph-mon[41444]: osdmap e327: 8 total, 8 up, 8 in 2024-02-05T07:41:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:03 smithi043 ceph-mon[41444]: pgmap v448: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:41:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4149899887' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:03 smithi043 ceph-mon[29879]: osdmap e327: 8 total, 8 up, 8 in 2024-02-05T07:41:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:03 smithi043 ceph-mon[29879]: pgmap v448: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:41:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4149899887' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81716-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:03 smithi050 ceph-mon[33492]: osdmap e327: 8 total, 8 up, 8 in 2024-02-05T07:41:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:03 smithi050 ceph-mon[33492]: pgmap v448: 324 pgs: 32 unknown, 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: osdmap e328: 8 total, 8 up, 8 in 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: osdmap e328: 8 total, 8 up, 8 in 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]: dispatch 2024-02-05T07:41:04.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]: dispatch 2024-02-05T07:41:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: osdmap e328: 8 total, 8 up, 8 in 2024-02-05T07:41:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]: dispatch 2024-02-05T07:41:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]: dispatch 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]': finished 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[41444]: osdmap e329: 8 total, 8 up, 8 in 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[41444]: pgmap v451: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]': finished 2024-02-05T07:41:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[29879]: osdmap e329: 8 total, 8 up, 8 in 2024-02-05T07:41:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:05 smithi043 ceph-mon[29879]: pgmap v451: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81716-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-45"}]': finished 2024-02-05T07:41:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:05 smithi050 ceph-mon[33492]: osdmap e329: 8 total, 8 up, 8 in 2024-02-05T07:41:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:05 smithi050 ceph-mon[33492]: pgmap v451: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:06 smithi043 ceph-mon[29879]: osdmap e330: 8 total, 8 up, 8 in 2024-02-05T07:41:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:06 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:06.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:06 smithi043 ceph-mon[41444]: osdmap e330: 8 total, 8 up, 8 in 2024-02-05T07:41:06.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:06 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:06 smithi050 ceph-mon[33492]: osdmap e330: 8 total, 8 up, 8 in 2024-02-05T07:41:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:06 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[29879]: osdmap e331: 8 total, 8 up, 8 in 2024-02-05T07:41:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[29879]: pgmap v454: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[41444]: osdmap e331: 8 total, 8 up, 8 in 2024-02-05T07:41:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:07 smithi043 ceph-mon[41444]: pgmap v454: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81716-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:07 smithi050 ceph-mon[33492]: osdmap e331: 8 total, 8 up, 8 in 2024-02-05T07:41:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:07 smithi050 ceph-mon[33492]: pgmap v454: 300 pgs: 40 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[29879]: osdmap e332: 8 total, 8 up, 8 in 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[41444]: osdmap e332: 8 total, 8 up, 8 in 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:08 smithi050 ceph-mon[33492]: osdmap e332: 8 total, 8 up, 8 in 2024-02-05T07:41:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:41:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[41444]: osdmap e333: 8 total, 8 up, 8 in 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[41444]: pgmap v457: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[29879]: osdmap e333: 8 total, 8 up, 8 in 2024-02-05T07:41:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:09 smithi043 ceph-mon[29879]: pgmap v457: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:09 smithi050 ceph-mon[33492]: osdmap e333: 8 total, 8 up, 8 in 2024-02-05T07:41:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:09.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:09.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:09 smithi050 ceph-mon[33492]: pgmap v457: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]': finished 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]: dispatch 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[29879]: osdmap e334: 8 total, 8 up, 8 in 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]: dispatch 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]': finished 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]: dispatch 2024-02-05T07:41:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[41444]: osdmap e334: 8 total, 8 up, 8 in 2024-02-05T07:41:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]: dispatch 2024-02-05T07:41:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-47"}]': finished 2024-02-05T07:41:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]: dispatch 2024-02-05T07:41:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-mon[33492]: osdmap e334: 8 total, 8 up, 8 in 2024-02-05T07:41:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2983297337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]: dispatch 2024-02-05T07:41:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]: dispatch 2024-02-05T07:41:11.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:41:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:41:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]': finished 2024-02-05T07:41:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: osdmap e335: 8 total, 8 up, 8 in 2024-02-05T07:41:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: pgmap v460: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: osdmap e336: 8 total, 8 up, 8 in 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81716-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]': finished 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: osdmap e335: 8 total, 8 up, 8 in 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: pgmap v460: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:11.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: osdmap e336: 8 total, 8 up, 8 in 2024-02-05T07:41:11.566 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81716-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-47", "mode": "writeback"}]': finished 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81716-61"}]': finished 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: osdmap e335: 8 total, 8 up, 8 in 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: pgmap v460: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:11.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: osdmap e336: 8 total, 8 up, 8 in 2024-02-05T07:41:11.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81716-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:41:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:41:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:41:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: osdmap e337: 8 total, 8 up, 8 in 2024-02-05T07:41:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: pgmap v463: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:41:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81716-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: osdmap e338: 8 total, 8 up, 8 in 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: osdmap e337: 8 total, 8 up, 8 in 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: pgmap v463: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:41:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:13.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81716-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:13.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:41:13.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: osdmap e338: 8 total, 8 up, 8 in 2024-02-05T07:41:13.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:13.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: osdmap e337: 8 total, 8 up, 8 in 2024-02-05T07:41:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: pgmap v463: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:41:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81716-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:41:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: osdmap e338: 8 total, 8 up, 8 in 2024-02-05T07:41:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:41:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[29879]: osdmap e339: 8 total, 8 up, 8 in 2024-02-05T07:41:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:41:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:41:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[29879]: pgmap v466: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:41:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[41444]: osdmap e339: 8 total, 8 up, 8 in 2024-02-05T07:41:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:41:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:41:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:15 smithi043 ceph-mon[41444]: pgmap v466: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:41:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:15 smithi050 ceph-mon[33492]: osdmap e339: 8 total, 8 up, 8 in 2024-02-05T07:41:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:41:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:41:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:15 smithi050 ceph-mon[33492]: pgmap v466: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[29879]: osdmap e340: 8 total, 8 up, 8 in 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:41:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:41:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[41444]: osdmap e340: 8 total, 8 up, 8 in 2024-02-05T07:41:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:41:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:16 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:41:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:41:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:16 smithi050 ceph-mon[33492]: osdmap e340: 8 total, 8 up, 8 in 2024-02-05T07:41:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:41:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: osdmap e341: 8 total, 8 up, 8 in 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: pgmap v469: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: osdmap e341: 8 total, 8 up, 8 in 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: pgmap v469: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:41:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: osdmap e341: 8 total, 8 up, 8 in 2024-02-05T07:41:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]: dispatch 2024-02-05T07:41:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: pgmap v469: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:17.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2636825894' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81716-62"}]': finished 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: osdmap e342: 8 total, 8 up, 8 in 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81716-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]': finished 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81716-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: osdmap e343: 8 total, 8 up, 8 in 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81716-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: osdmap e342: 8 total, 8 up, 8 in 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81716-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]': finished 2024-02-05T07:41:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81716-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: osdmap e343: 8 total, 8 up, 8 in 2024-02-05T07:41:18.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81716-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: osdmap e342: 8 total, 8 up, 8 in 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81716-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]': finished 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81716-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: osdmap e343: 8 total, 8 up, 8 in 2024-02-05T07:41:18.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81716-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:41:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:41:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:20.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[29879]: pgmap v472: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:20.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:20.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:20.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:20.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:20.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:19 smithi043 ceph-mon[41444]: pgmap v472: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:20.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:20.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:20.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:20.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-47"}]: dispatch 2024-02-05T07:41:20.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:19 smithi050 ceph-mon[33492]: pgmap v472: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: osdmap e344: 8 total, 8 up, 8 in 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81716-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: osdmap e345: 8 total, 8 up, 8 in 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:21.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: osdmap e344: 8 total, 8 up, 8 in 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81716-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: osdmap e345: 8 total, 8 up, 8 in 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:21.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:21.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:41:21.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:41:21.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: osdmap e344: 8 total, 8 up, 8 in 2024-02-05T07:41:21.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:41:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:41:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81716-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: osdmap e345: 8 total, 8 up, 8 in 2024-02-05T07:41:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:21.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:22.062 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[41444]: pgmap v475: 300 pgs: 40 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[41444]: osdmap e346: 8 total, 8 up, 8 in 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[29879]: pgmap v475: 300 pgs: 40 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:22.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:22.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[29879]: osdmap e346: 8 total, 8 up, 8 in 2024-02-05T07:41:22.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:22.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:22.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:21 smithi050 ceph-mon[33492]: pgmap v475: 300 pgs: 40 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:41:22.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:22.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:21 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:22.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:22.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:21 smithi050 ceph-mon[33492]: osdmap e346: 8 total, 8 up, 8 in 2024-02-05T07:41:22.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:22.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[41444]: osdmap e347: 8 total, 8 up, 8 in 2024-02-05T07:41:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[41444]: pgmap v478: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:41:23.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:23.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[29879]: osdmap e347: 8 total, 8 up, 8 in 2024-02-05T07:41:23.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:23.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:23.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:23.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:23 smithi043 ceph-mon[29879]: pgmap v478: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:41:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:23 smithi050 ceph-mon[33492]: osdmap e347: 8 total, 8 up, 8 in 2024-02-05T07:41:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:23 smithi050 ceph-mon[33492]: pgmap v478: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]': finished 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: osdmap e348: 8 total, 8 up, 8 in 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]': finished 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: osdmap e348: 8 total, 8 up, 8 in 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-49"}]': finished 2024-02-05T07:41:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: osdmap e348: 8 total, 8 up, 8 in 2024-02-05T07:41:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:41:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]: dispatch 2024-02-05T07:41:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]: dispatch 2024-02-05T07:41:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1987118792' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81716-63"}]': finished 2024-02-05T07:41:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]': finished 2024-02-05T07:41:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[29879]: osdmap e349: 8 total, 8 up, 8 in 2024-02-05T07:41:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81716-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[29879]: pgmap v481: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]': finished 2024-02-05T07:41:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[41444]: osdmap e349: 8 total, 8 up, 8 in 2024-02-05T07:41:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81716-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:25 smithi043 ceph-mon[41444]: pgmap v481: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-49", "mode": "readproxy"}]': finished 2024-02-05T07:41:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:25 smithi050 ceph-mon[33492]: osdmap e349: 8 total, 8 up, 8 in 2024-02-05T07:41:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:25.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:25.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81716-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:25.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:25 smithi050 ceph-mon[33492]: pgmap v481: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81716-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:26 smithi043 ceph-mon[41444]: osdmap e350: 8 total, 8 up, 8 in 2024-02-05T07:41:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81716-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81716-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:26 smithi043 ceph-mon[29879]: osdmap e350: 8 total, 8 up, 8 in 2024-02-05T07:41:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81716-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81716-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:26 smithi050 ceph-mon[33492]: osdmap e350: 8 total, 8 up, 8 in 2024-02-05T07:41:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81716-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:27 smithi043 ceph-mon[29879]: osdmap e351: 8 total, 8 up, 8 in 2024-02-05T07:41:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:27 smithi043 ceph-mon[29879]: pgmap v484: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:41:27.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:27 smithi043 ceph-mon[41444]: osdmap e351: 8 total, 8 up, 8 in 2024-02-05T07:41:27.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:27 smithi043 ceph-mon[41444]: pgmap v484: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:41:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:27 smithi050 ceph-mon[33492]: osdmap e351: 8 total, 8 up, 8 in 2024-02-05T07:41:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:27 smithi050 ceph-mon[33492]: pgmap v484: 292 pgs: 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:41:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81716-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:28 smithi043 ceph-mon[29879]: osdmap e352: 8 total, 8 up, 8 in 2024-02-05T07:41:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81716-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:28 smithi043 ceph-mon[41444]: osdmap e352: 8 total, 8 up, 8 in 2024-02-05T07:41:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81716-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:28 smithi050 ceph-mon[33492]: osdmap e352: 8 total, 8 up, 8 in 2024-02-05T07:41:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:29 smithi043 ceph-mon[41444]: osdmap e353: 8 total, 8 up, 8 in 2024-02-05T07:41:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:29 smithi043 ceph-mon[41444]: pgmap v487: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:41:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:29 smithi043 ceph-mon[29879]: osdmap e353: 8 total, 8 up, 8 in 2024-02-05T07:41:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:29 smithi043 ceph-mon[29879]: pgmap v487: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:41:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:41:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:41:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:41:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:29 smithi050 ceph-mon[33492]: osdmap e353: 8 total, 8 up, 8 in 2024-02-05T07:41:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:29 smithi050 ceph-mon[33492]: pgmap v487: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:41:30.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:30 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:30 smithi043 ceph-mon[41444]: osdmap e354: 8 total, 8 up, 8 in 2024-02-05T07:41:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:30 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:30 smithi043 ceph-mon[29879]: osdmap e354: 8 total, 8 up, 8 in 2024-02-05T07:41:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:30 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:30 smithi050 ceph-mon[33492]: osdmap e354: 8 total, 8 up, 8 in 2024-02-05T07:41:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:31.318 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:41:30 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:41:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:31 smithi050 ceph-mon[33492]: osdmap e355: 8 total, 8 up, 8 in 2024-02-05T07:41:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:31 smithi050 ceph-mon[33492]: pgmap v490: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:31 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:31.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[41444]: osdmap e355: 8 total, 8 up, 8 in 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[41444]: pgmap v490: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[29879]: osdmap e355: 8 total, 8 up, 8 in 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]: dispatch 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[29879]: pgmap v490: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/89351809' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81716-64"}]': finished 2024-02-05T07:41:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:32 smithi050 ceph-mon[33492]: osdmap e356: 8 total, 8 up, 8 in 2024-02-05T07:41:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[29879]: osdmap e356: 8 total, 8 up, 8 in 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[41444]: osdmap e356: 8 total, 8 up, 8 in 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:32.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:32.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[41444]: osdmap e357: 8 total, 8 up, 8 in 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[41444]: pgmap v493: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[29879]: osdmap e357: 8 total, 8 up, 8 in 2024-02-05T07:41:33.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:33.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:33.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:33 smithi043 ceph-mon[29879]: pgmap v493: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81716-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:33 smithi050 ceph-mon[33492]: osdmap e357: 8 total, 8 up, 8 in 2024-02-05T07:41:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:33 smithi050 ceph-mon[33492]: pgmap v493: 292 pgs: 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:34 smithi043 ceph-mon[29879]: osdmap e358: 8 total, 8 up, 8 in 2024-02-05T07:41:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:34 smithi043 ceph-mon[41444]: osdmap e358: 8 total, 8 up, 8 in 2024-02-05T07:41:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:34 smithi050 ceph-mon[33492]: osdmap e358: 8 total, 8 up, 8 in 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[29879]: osdmap e359: 8 total, 8 up, 8 in 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[29879]: pgmap v496: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[41444]: osdmap e359: 8 total, 8 up, 8 in 2024-02-05T07:41:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:35 smithi043 ceph-mon[41444]: pgmap v496: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81716-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:35 smithi050 ceph-mon[33492]: osdmap e359: 8 total, 8 up, 8 in 2024-02-05T07:41:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:35 smithi050 ceph-mon[33492]: pgmap v496: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[29879]: osdmap e360: 8 total, 8 up, 8 in 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[41444]: osdmap e360: 8 total, 8 up, 8 in 2024-02-05T07:41:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:36 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:36 smithi050 ceph-mon[33492]: osdmap e360: 8 total, 8 up, 8 in 2024-02-05T07:41:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:37.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP23:head 2024-02-05T07:41:37.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:5d165639:::164:head 2024-02-05T07:41:37.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f43765fc:::165:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b4c720e9:::166:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e694b040:::167:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:afa38db2:::168:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:77ba9f53:::169:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:87495034:::170:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7c96bf0e:::171:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:dbe346cc:::172:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:e943ec24:::173:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f97a9c0c:::174:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:6f26e74d:::175:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4f95e106:::176:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:0e6f2f8f:::177:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:05db05f1:::178:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:38a78d66:::179:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:d095610b:::180:head 2024-02-05T07:41:37.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:a1a9d709:::181:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1e5d39db:::182:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:f7df4fb9:::183:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:03a7f161:::184:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ba70721e:::185:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:28e5662d:::186:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:973d52de:::187:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:4303eb1c:::188:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:b990b48e:::189:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:29b8165b:::190:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:3547f197:::191:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:7e260936:::192:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:1abec7b1:::193:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:10fdda93:::194:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:15817eea:::195:head 2024-02-05T07:41:37.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:770bab57:::196:head 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:ed9e13e7:::197:head 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:71471a8f:::198:head 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 276:10fb1d02:::199:head 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8025 ms) 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1707118852 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,1707118857,1707118858,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,1707118857,1707118858,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,1707118857,1707118858,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,1707118857,1707118858,1707118860,1707118861,0 2024-02-05T07:41:37.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,1707118857,1707118858,1707118860,1707118861,0 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118852,1707118854,1707118855,1707118857,1707118858,1707118860,1707118861,0 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707118855,1707118857,1707118858,1707118860,1707118861,1707118863,1707118864,0 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1707118855, trimmed 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21375 ms) 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14052 ms) 2024-02-05T07:41:37.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]': finished 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: osdmap e361: 8 total, 8 up, 8 in 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[41444]: pgmap v499: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]': finished 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: osdmap e361: 8 total, 8 up, 8 in 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:37 smithi043 ceph-mon[29879]: pgmap v499: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]': finished 2024-02-05T07:41:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: osdmap e361: 8 total, 8 up, 8 in 2024-02-05T07:41:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-49"}]: dispatch 2024-02-05T07:41:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:37 smithi050 ceph-mon[33492]: pgmap v499: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[29879]: osdmap e362: 8 total, 8 up, 8 in 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[41444]: osdmap e362: 8 total, 8 up, 8 in 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:38 smithi050 ceph-mon[33492]: osdmap e362: 8 total, 8 up, 8 in 2024-02-05T07:41:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/645935192' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]: dispatch 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:41:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: osdmap e363: 8 total, 8 up, 8 in 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81716-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[29879]: pgmap v502: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: osdmap e363: 8 total, 8 up, 8 in 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81716-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:39 smithi043 ceph-mon[41444]: pgmap v502: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81716-65"}]': finished 2024-02-05T07:41:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: osdmap e363: 8 total, 8 up, 8 in 2024-02-05T07:41:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:41:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:39.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:39.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81716-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:41:39.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:39 smithi050 ceph-mon[33492]: pgmap v502: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:41:40.982 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:40.982 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81716-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:40.982 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:40 smithi050 ceph-mon[33492]: osdmap e364: 8 total, 8 up, 8 in 2024-02-05T07:41:40.982 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81716-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:40.982 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:40.982 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81716-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[29879]: osdmap e364: 8 total, 8 up, 8 in 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81716-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81716-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[41444]: osdmap e364: 8 total, 8 up, 8 in 2024-02-05T07:41:41.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81716-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:41.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:41.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:41:41.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:41:40 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:41:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:42.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: osdmap e365: 8 total, 8 up, 8 in 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: pgmap v505: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81716-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]': finished 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]: dispatch 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: osdmap e366: 8 total, 8 up, 8 in 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]: dispatch 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: osdmap e365: 8 total, 8 up, 8 in 2024-02-05T07:41:42.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:42.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: pgmap v505: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:41:42.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:42.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81716-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:42.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]': finished 2024-02-05T07:41:42.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]: dispatch 2024-02-05T07:41:42.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: osdmap e366: 8 total, 8 up, 8 in 2024-02-05T07:41:42.065 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]: dispatch 2024-02-05T07:41:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:41:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: osdmap e365: 8 total, 8 up, 8 in 2024-02-05T07:41:42.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: pgmap v505: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:41:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81716-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-51"}]': finished 2024-02-05T07:41:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]: dispatch 2024-02-05T07:41:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: osdmap e366: 8 total, 8 up, 8 in 2024-02-05T07:41:42.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]: dispatch 2024-02-05T07:41:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]': finished 2024-02-05T07:41:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[29879]: osdmap e367: 8 total, 8 up, 8 in 2024-02-05T07:41:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:43.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]': finished 2024-02-05T07:41:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[41444]: osdmap e367: 8 total, 8 up, 8 in 2024-02-05T07:41:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:43.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:42 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:42 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:41:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:42 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:41:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-51", "mode": "writeback"}]': finished 2024-02-05T07:41:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:42 smithi050 ceph-mon[33492]: osdmap e367: 8 total, 8 up, 8 in 2024-02-05T07:41:43.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:41:43.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:42 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[41444]: pgmap v508: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[41444]: osdmap e368: 8 total, 8 up, 8 in 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[29879]: pgmap v508: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[29879]: osdmap e368: 8 total, 8 up, 8 in 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:44.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:44.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:43 smithi050 ceph-mon[33492]: pgmap v508: 300 pgs: 8 unknown, 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:41:44.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:41:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:43 smithi050 ceph-mon[33492]: osdmap e368: 8 total, 8 up, 8 in 2024-02-05T07:41:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:41:44.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:45.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ 2024-02-05T07:41:45.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3023 ms) 2024-02-05T07:41:45.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-05T07:41:45.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7048 ms) 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7012 ms) 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7016 ms) 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7042 ms) 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7062 ms) 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7019 ms) 2024-02-05T07:41:45.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139384 ms total) 2024-02-05T07:41:45.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-02-05T07:41:45.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-05T07:41:45.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (301737 ms total) 2024-02-05T07:41:45.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[29879]: osdmap e369: 8 total, 8 up, 8 in 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[29879]: pgmap v511: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:41:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[41444]: osdmap e369: 8 total, 8 up, 8 in 2024-02-05T07:41:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:45 smithi043 ceph-mon[41444]: pgmap v511: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:41:45.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:45.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:45 smithi050 ceph-mon[33492]: osdmap e369: 8 total, 8 up, 8 in 2024-02-05T07:41:45.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]: dispatch 2024-02-05T07:41:45.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:45.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:41:45.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:45 smithi050 ceph-mon[33492]: pgmap v511: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[29879]: osdmap e370: 8 total, 8 up, 8 in 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:41:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[41444]: osdmap e370: 8 total, 8 up, 8 in 2024-02-05T07:41:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:46 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:41:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4018815831' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81716-66"}]': finished 2024-02-05T07:41:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:41:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:46 smithi050 ceph-mon[33492]: osdmap e370: 8 total, 8 up, 8 in 2024-02-05T07:41:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:41:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:41:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:41:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[41444]: osdmap e371: 8 total, 8 up, 8 in 2024-02-05T07:41:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:41:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[41444]: pgmap v514: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:41:47.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:41:47.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[29879]: osdmap e371: 8 total, 8 up, 8 in 2024-02-05T07:41:47.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:41:47.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:47 smithi043 ceph-mon[29879]: pgmap v514: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:41:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:41:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:47 smithi050 ceph-mon[33492]: osdmap e371: 8 total, 8 up, 8 in 2024-02-05T07:41:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:41:47.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:47 smithi050 ceph-mon[33492]: pgmap v514: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:41:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:48 smithi043 ceph-mon[41444]: osdmap e372: 8 total, 8 up, 8 in 2024-02-05T07:41:48.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:41:48.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:48 smithi043 ceph-mon[29879]: osdmap e372: 8 total, 8 up, 8 in 2024-02-05T07:41:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-51","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:41:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:48 smithi050 ceph-mon[33492]: osdmap e372: 8 total, 8 up, 8 in 2024-02-05T07:41:49.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:49 smithi043 ceph-mon[41444]: pgmap v516: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:49 smithi043 ceph-mon[29879]: pgmap v516: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:41:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:41:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:41:49.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:49 smithi050 ceph-mon[33492]: pgmap v516: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:50 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:41:50.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:50 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:41:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:50 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:41:51.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:41:50 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:41:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:52 smithi043 ceph-mon[41444]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:52 smithi043 ceph-mon[29879]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:52 smithi050 ceph-mon[33492]: pgmap v517: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:41:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:41:52.431 INFO:tasks.workunit.client.0.smithi043.stdout: started 0x55567c1adc90 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1addc0 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aded0 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae280 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae6f0 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae800 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aedf0 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae390 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae4e0 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae950 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aea80 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aebb0 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1aef40 2024-02-05T07:41:52.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af0b0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ae000 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af1e0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af670 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af7c0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ab950 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1abae0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1afdb0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af910 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1afa40 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1afb70 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1afca0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0360 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1afec0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1afff0 2024-02-05T07:41:52.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0100 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0210 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0930 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0490 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b05a0 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b06f0 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0820 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0f20 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0a80 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0bd0 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0ce0 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1050 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b14e0 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1630 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b0e10 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1ac0 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1760 2024-02-05T07:41:52.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b18b0 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af2f0 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af400 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1af530 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2070 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1bd0 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1d20 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1e30 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1f60 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2660 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b21c0 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2310 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2420 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b27b0 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2c20 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2d70 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b1160 2024-02-05T07:41:52.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b12b0 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3340 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2ea0 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2ff0 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3100 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3230 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3910 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3470 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b35a0 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b36b0 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3a40 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3eb0 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4000 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b45b0 2024-02-05T07:41:52.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3b50 2024-02-05T07:41:52.437 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b3c80 2024-02-05T07:41:52.437 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4110 2024-02-05T07:41:52.437 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4240 2024-02-05T07:41:52.437 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4350 2024-02-05T07:41:52.437 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4460 2024-02-05T07:41:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:54 smithi043 ceph-mon[29879]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:41:54.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:54 smithi043 ceph-mon[41444]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:41:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:54 smithi050 ceph-mon[33492]: pgmap v518: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:41:56.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:56 smithi043 ceph-mon[29879]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 612 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:41:56.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:56 smithi043 ceph-mon[41444]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 612 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:41:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:56 smithi050 ceph-mon[33492]: pgmap v519: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 612 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[41444]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[29879]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:58 smithi050 ceph-mon[33492]: pgmap v520: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:41:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:41:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:41:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:41:59.312 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:41:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[29879]: osdmap e373: 8 total, 8 up, 8 in 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]': finished 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[29879]: osdmap e374: 8 total, 8 up, 8 in 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:59.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[41444]: osdmap e373: 8 total, 8 up, 8 in 2024-02-05T07:41:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]': finished 2024-02-05T07:41:59.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:41:59 smithi043 ceph-mon[41444]: osdmap e374: 8 total, 8 up, 8 in 2024-02-05T07:41:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:41:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:59 smithi050 ceph-mon[33492]: osdmap e373: 8 total, 8 up, 8 in 2024-02-05T07:41:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:41:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]': finished 2024-02-05T07:41:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:41:59 smithi050 ceph-mon[33492]: osdmap e374: 8 total, 8 up, 8 in 2024-02-05T07:42:00.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[29879]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[41444]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:42:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:42:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:00 smithi050 ceph-mon[33492]: pgmap v522: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:42:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-51"}]: dispatch 2024-02-05T07:42:01.311 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:42:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[29879]: osdmap e375: 8 total, 8 up, 8 in 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[29879]: pgmap v525: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[29879]: osdmap e376: 8 total, 8 up, 8 in 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[41444]: osdmap e375: 8 total, 8 up, 8 in 2024-02-05T07:42:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[41444]: pgmap v525: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:42:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[41444]: osdmap e376: 8 total, 8 up, 8 in 2024-02-05T07:42:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:01 smithi050 ceph-mon[33492]: osdmap e375: 8 total, 8 up, 8 in 2024-02-05T07:42:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:01 smithi050 ceph-mon[33492]: pgmap v525: 260 pgs: 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:42:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:01 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:01 smithi050 ceph-mon[33492]: osdmap e376: 8 total, 8 up, 8 in 2024-02-05T07:42:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[29879]: osdmap e377: 8 total, 8 up, 8 in 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-53"}]: dispatch 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-53"}]: dispatch 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[41444]: osdmap e377: 8 total, 8 up, 8 in 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-53"}]: dispatch 2024-02-05T07:42:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-53"}]: dispatch 2024-02-05T07:42:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:02 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:42:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:02 smithi050 ceph-mon[33492]: osdmap e377: 8 total, 8 up, 8 in 2024-02-05T07:42:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-53"}]: dispatch 2024-02-05T07:42:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-53"}]: dispatch 2024-02-05T07:42:04.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:04 smithi043 ceph-mon[29879]: pgmap v528: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:42:04.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:04 smithi043 ceph-mon[29879]: osdmap e378: 8 total, 8 up, 8 in 2024-02-05T07:42:04.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:04 smithi043 ceph-mon[41444]: pgmap v528: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:42:04.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:04 smithi043 ceph-mon[41444]: osdmap e378: 8 total, 8 up, 8 in 2024-02-05T07:42:04.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:04 smithi050 ceph-mon[33492]: pgmap v528: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:42:04.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:04 smithi050 ceph-mon[33492]: osdmap e378: 8 total, 8 up, 8 in 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[29879]: osdmap e379: 8 total, 8 up, 8 in 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[41444]: osdmap e379: 8 total, 8 up, 8 in 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:05 smithi050 ceph-mon[33492]: osdmap e379: 8 total, 8 up, 8 in 2024-02-05T07:42:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: pgmap v531: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: osdmap e380: 8 total, 8 up, 8 in 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-55"}]: dispatch 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-55"}]: dispatch 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[29879]: osdmap e381: 8 total, 8 up, 8 in 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: pgmap v531: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: osdmap e380: 8 total, 8 up, 8 in 2024-02-05T07:42:06.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:06.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:42:06.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:06.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:06.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-55"}]: dispatch 2024-02-05T07:42:06.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-55"}]: dispatch 2024-02-05T07:42:06.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:06 smithi043 ceph-mon[41444]: osdmap e381: 8 total, 8 up, 8 in 2024-02-05T07:42:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: pgmap v531: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: osdmap e380: 8 total, 8 up, 8 in 2024-02-05T07:42:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:42:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:06.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:06.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-55"}]: dispatch 2024-02-05T07:42:06.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-55"}]: dispatch 2024-02-05T07:42:06.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:06 smithi050 ceph-mon[33492]: osdmap e381: 8 total, 8 up, 8 in 2024-02-05T07:42:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[41444]: pgmap v534: 260 pgs: 260 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:42:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[41444]: osdmap e382: 8 total, 8 up, 8 in 2024-02-05T07:42:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[29879]: pgmap v534: 260 pgs: 260 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:42:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[29879]: osdmap e382: 8 total, 8 up, 8 in 2024-02-05T07:42:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:08 smithi050 ceph-mon[33492]: pgmap v534: 260 pgs: 260 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:42:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:08 smithi050 ceph-mon[33492]: osdmap e382: 8 total, 8 up, 8 in 2024-02-05T07:42:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:42:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:42:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[41444]: osdmap e383: 8 total, 8 up, 8 in 2024-02-05T07:42:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-57"}]: dispatch 2024-02-05T07:42:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-57"}]: dispatch 2024-02-05T07:42:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[29879]: osdmap e383: 8 total, 8 up, 8 in 2024-02-05T07:42:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-57"}]: dispatch 2024-02-05T07:42:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-57"}]: dispatch 2024-02-05T07:42:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:09 smithi050 ceph-mon[33492]: osdmap e383: 8 total, 8 up, 8 in 2024-02-05T07:42:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-57"}]: dispatch 2024-02-05T07:42:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-57"}]: dispatch 2024-02-05T07:42:10.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:10 smithi043 ceph-mon[29879]: pgmap v537: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:42:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:10 smithi043 ceph-mon[29879]: osdmap e384: 8 total, 8 up, 8 in 2024-02-05T07:42:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:10 smithi043 ceph-mon[41444]: pgmap v537: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:42:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:10 smithi043 ceph-mon[41444]: osdmap e384: 8 total, 8 up, 8 in 2024-02-05T07:42:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:10 smithi050 ceph-mon[33492]: pgmap v537: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-05T07:42:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:10 smithi050 ceph-mon[33492]: osdmap e384: 8 total, 8 up, 8 in 2024-02-05T07:42:11.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:42:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:42:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:11 smithi050 ceph-mon[33492]: osdmap e385: 8 total, 8 up, 8 in 2024-02-05T07:42:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:11.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:11 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[41444]: osdmap e385: 8 total, 8 up, 8 in 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[29879]: osdmap e385: 8 total, 8 up, 8 in 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:11 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: pgmap v540: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: osdmap e386: 8 total, 8 up, 8 in 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-59"}]: dispatch 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-59"}]: dispatch 2024-02-05T07:42:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[29879]: osdmap e387: 8 total, 8 up, 8 in 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: pgmap v540: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: osdmap e386: 8 total, 8 up, 8 in 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-59"}]: dispatch 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-59"}]: dispatch 2024-02-05T07:42:12.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:12 smithi043 ceph-mon[41444]: osdmap e387: 8 total, 8 up, 8 in 2024-02-05T07:42:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: pgmap v540: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:42:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: osdmap e386: 8 total, 8 up, 8 in 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-59"}]: dispatch 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-59"}]: dispatch 2024-02-05T07:42:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:12 smithi050 ceph-mon[33492]: osdmap e387: 8 total, 8 up, 8 in 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[29879]: pgmap v543: 260 pgs: 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[29879]: osdmap e388: 8 total, 8 up, 8 in 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[41444]: pgmap v543: 260 pgs: 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[41444]: osdmap e388: 8 total, 8 up, 8 in 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:14.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:14.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:13 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:42:14.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:14.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:13 smithi050 ceph-mon[33492]: pgmap v543: 260 pgs: 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-02-05T07:42:14.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:13 smithi050 ceph-mon[33492]: osdmap e388: 8 total, 8 up, 8 in 2024-02-05T07:42:14.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:14.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[29879]: osdmap e389: 8 total, 8 up, 8 in 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[41444]: osdmap e389: 8 total, 8 up, 8 in 2024-02-05T07:42:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:15 smithi050 ceph-mon[33492]: osdmap e389: 8 total, 8 up, 8 in 2024-02-05T07:42:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[29879]: pgmap v546: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[29879]: osdmap e390: 8 total, 8 up, 8 in 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-61"}]: dispatch 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-61"}]: dispatch 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[41444]: pgmap v546: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:42:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[41444]: osdmap e390: 8 total, 8 up, 8 in 2024-02-05T07:42:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-61"}]: dispatch 2024-02-05T07:42:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-61"}]: dispatch 2024-02-05T07:42:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:16 smithi050 ceph-mon[33492]: pgmap v546: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:42:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:16 smithi050 ceph-mon[33492]: osdmap e390: 8 total, 8 up, 8 in 2024-02-05T07:42:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-61"}]: dispatch 2024-02-05T07:42:16.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-61"}]: dispatch 2024-02-05T07:42:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[41444]: osdmap e391: 8 total, 8 up, 8 in 2024-02-05T07:42:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[41444]: pgmap v549: 260 pgs: 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:42:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[41444]: osdmap e392: 8 total, 8 up, 8 in 2024-02-05T07:42:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:17.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[29879]: osdmap e391: 8 total, 8 up, 8 in 2024-02-05T07:42:17.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[29879]: pgmap v549: 260 pgs: 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:42:17.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[29879]: osdmap e392: 8 total, 8 up, 8 in 2024-02-05T07:42:17.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:17.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:17 smithi050 ceph-mon[33492]: osdmap e391: 8 total, 8 up, 8 in 2024-02-05T07:42:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:17 smithi050 ceph-mon[33492]: pgmap v549: 260 pgs: 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:42:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:17 smithi050 ceph-mon[33492]: osdmap e392: 8 total, 8 up, 8 in 2024-02-05T07:42:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:17.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:42:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:42:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[41444]: osdmap e393: 8 total, 8 up, 8 in 2024-02-05T07:42:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[29879]: osdmap e393: 8 total, 8 up, 8 in 2024-02-05T07:42:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:19 smithi050 ceph-mon[33492]: osdmap e393: 8 total, 8 up, 8 in 2024-02-05T07:42:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:20.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[29879]: pgmap v552: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:42:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:42:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[29879]: osdmap e394: 8 total, 8 up, 8 in 2024-02-05T07:42:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:42:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[41444]: pgmap v552: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:42:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:42:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[41444]: osdmap e394: 8 total, 8 up, 8 in 2024-02-05T07:42:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:20 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:42:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:20 smithi050 ceph-mon[33492]: pgmap v552: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:42:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:42:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:20 smithi050 ceph-mon[33492]: osdmap e394: 8 total, 8 up, 8 in 2024-02-05T07:42:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:20 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:42:21.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:42:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:42:21.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:21 smithi050 ceph-mon[33492]: osdmap e395: 8 total, 8 up, 8 in 2024-02-05T07:42:21.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:21 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:42:21.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:21 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:42:21.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:21 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:42:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[29879]: osdmap e395: 8 total, 8 up, 8 in 2024-02-05T07:42:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:42:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:42:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:42:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[41444]: osdmap e395: 8 total, 8 up, 8 in 2024-02-05T07:42:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:42:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:42:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:21 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:42:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:22 smithi043 ceph-mon[29879]: pgmap v555: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:42:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:22 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:22 smithi043 ceph-mon[29879]: osdmap e396: 8 total, 8 up, 8 in 2024-02-05T07:42:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:22 smithi043 ceph-mon[41444]: pgmap v555: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:42:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:22 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:22 smithi043 ceph-mon[41444]: osdmap e396: 8 total, 8 up, 8 in 2024-02-05T07:42:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:22 smithi050 ceph-mon[33492]: pgmap v555: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:42:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:22 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:22 smithi050 ceph-mon[33492]: osdmap e396: 8 total, 8 up, 8 in 2024-02-05T07:42:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:23 smithi043 ceph-mon[29879]: osdmap e397: 8 total, 8 up, 8 in 2024-02-05T07:42:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:23 smithi043 ceph-mon[41444]: osdmap e397: 8 total, 8 up, 8 in 2024-02-05T07:42:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:23 smithi050 ceph-mon[33492]: osdmap e397: 8 total, 8 up, 8 in 2024-02-05T07:42:24.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:24 smithi043 ceph-mon[29879]: pgmap v558: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:42:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:24 smithi043 ceph-mon[29879]: osdmap e398: 8 total, 8 up, 8 in 2024-02-05T07:42:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:24 smithi043 ceph-mon[41444]: pgmap v558: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:42:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:24 smithi043 ceph-mon[41444]: osdmap e398: 8 total, 8 up, 8 in 2024-02-05T07:42:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:24 smithi050 ceph-mon[33492]: pgmap v558: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-02-05T07:42:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:24 smithi050 ceph-mon[33492]: osdmap e398: 8 total, 8 up, 8 in 2024-02-05T07:42:26.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:26 smithi043 ceph-mon[29879]: pgmap v560: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:42:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:26 smithi043 ceph-mon[41444]: pgmap v560: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:42:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:26 smithi050 ceph-mon[33492]: pgmap v560: 292 pgs: 292 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:42:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:28 smithi043 ceph-mon[41444]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-02-05T07:42:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:28 smithi043 ceph-mon[29879]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-02-05T07:42:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:28 smithi050 ceph-mon[33492]: pgmap v561: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-02-05T07:42:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:42:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:42:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:42:30.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:30 smithi043 ceph-mon[29879]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 536 B/s wr, 2 op/s 2024-02-05T07:42:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:30 smithi043 ceph-mon[41444]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 536 B/s wr, 2 op/s 2024-02-05T07:42:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:30 smithi050 ceph-mon[33492]: pgmap v562: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 536 B/s wr, 2 op/s 2024-02-05T07:42:31.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:42:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:42:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:32 smithi043 ceph-mon[29879]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 474 B/s wr, 2 op/s 2024-02-05T07:42:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:32.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:32 smithi043 ceph-mon[29879]: osdmap e399: 8 total, 8 up, 8 in 2024-02-05T07:42:32.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:32 smithi043 ceph-mon[41444]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 474 B/s wr, 2 op/s 2024-02-05T07:42:32.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:32.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:32 smithi043 ceph-mon[41444]: osdmap e399: 8 total, 8 up, 8 in 2024-02-05T07:42:32.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:32 smithi050 ceph-mon[33492]: pgmap v563: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 474 B/s wr, 2 op/s 2024-02-05T07:42:32.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:32.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:32 smithi050 ceph-mon[33492]: osdmap e399: 8 total, 8 up, 8 in 2024-02-05T07:42:34.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:34 smithi043 ceph-mon[29879]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 425 B/s wr, 2 op/s 2024-02-05T07:42:34.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:34 smithi043 ceph-mon[41444]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 425 B/s wr, 2 op/s 2024-02-05T07:42:34.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:34 smithi050 ceph-mon[33492]: pgmap v565: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 425 B/s wr, 2 op/s 2024-02-05T07:42:35.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:35 smithi043 ceph-mon[29879]: osdmap e400: 8 total, 8 up, 8 in 2024-02-05T07:42:35.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:35 smithi043 ceph-mon[41444]: osdmap e400: 8 total, 8 up, 8 in 2024-02-05T07:42:35.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:35 smithi050 ceph-mon[33492]: osdmap e400: 8 total, 8 up, 8 in 2024-02-05T07:42:36.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:36 smithi043 ceph-mon[29879]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:42:36.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:36 smithi043 ceph-mon[41444]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:42:36.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:36 smithi050 ceph-mon[33492]: pgmap v567: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:42:38.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:38 smithi043 ceph-mon[29879]: pgmap v568: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:38.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:38 smithi043 ceph-mon[41444]: pgmap v568: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:38.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:38 smithi050 ceph-mon[33492]: pgmap v568: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:42:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:42:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:42:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:40 smithi043 ceph-mon[29879]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:40.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:40 smithi043 ceph-mon[41444]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:40 smithi050 ceph-mon[33492]: pgmap v569: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:41.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:42:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:42:42.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:42 smithi043 ceph-mon[41444]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:42.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:42 smithi043 ceph-mon[41444]: osdmap e401: 8 total, 8 up, 8 in 2024-02-05T07:42:42.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:42.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:42 smithi043 ceph-mon[29879]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:42.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:42 smithi043 ceph-mon[29879]: osdmap e401: 8 total, 8 up, 8 in 2024-02-05T07:42:42.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:42.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:42 smithi050 ceph-mon[33492]: pgmap v570: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:42.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:42 smithi050 ceph-mon[33492]: osdmap e401: 8 total, 8 up, 8 in 2024-02-05T07:42:42.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:43.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:43.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:43.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:44.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:44 smithi043 ceph-mon[41444]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:44.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:44 smithi043 ceph-mon[29879]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:44.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:44 smithi050 ceph-mon[33492]: pgmap v572: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:46 smithi043 ceph-mon[29879]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:46 smithi043 ceph-mon[29879]: osdmap e402: 8 total, 8 up, 8 in 2024-02-05T07:42:46.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:46 smithi043 ceph-mon[41444]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:46.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:46 smithi043 ceph-mon[41444]: osdmap e402: 8 total, 8 up, 8 in 2024-02-05T07:42:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:46 smithi050 ceph-mon[33492]: pgmap v573: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:42:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:46 smithi050 ceph-mon[33492]: osdmap e402: 8 total, 8 up, 8 in 2024-02-05T07:42:48.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:48 smithi043 ceph-mon[29879]: pgmap v575: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:42:48.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:48 smithi043 ceph-mon[41444]: pgmap v575: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:42:48.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:48 smithi050 ceph-mon[33492]: pgmap v575: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:42:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:42:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:42:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:42:50.312 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:50 smithi043 ceph-mon[41444]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:42:50.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:50 smithi043 ceph-mon[29879]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:42:50.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:50 smithi050 ceph-mon[33492]: pgmap v576: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:42:51.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:42:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:42:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:52 smithi043 ceph-mon[29879]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:42:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:52 smithi043 ceph-mon[29879]: osdmap e403: 8 total, 8 up, 8 in 2024-02-05T07:42:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:52 smithi043 ceph-mon[41444]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:42:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:52 smithi043 ceph-mon[41444]: osdmap e403: 8 total, 8 up, 8 in 2024-02-05T07:42:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:52 smithi050 ceph-mon[33492]: pgmap v577: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:42:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:52 smithi050 ceph-mon[33492]: osdmap e403: 8 total, 8 up, 8 in 2024-02-05T07:42:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:42:54.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:54 smithi043 ceph-mon[29879]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:42:54.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:54 smithi043 ceph-mon[41444]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:42:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:54 smithi050 ceph-mon[33492]: pgmap v579: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[29879]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-63"}]: dispatch 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-63"}]: dispatch 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[41444]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-63"}]: dispatch 2024-02-05T07:42:56.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-63"}]: dispatch 2024-02-05T07:42:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:56 smithi050 ceph-mon[33492]: pgmap v580: 292 pgs: 292 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:42:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:42:56.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-63"}]: dispatch 2024-02-05T07:42:56.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-63"}]: dispatch 2024-02-05T07:42:57.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:57 smithi043 ceph-mon[29879]: osdmap e404: 8 total, 8 up, 8 in 2024-02-05T07:42:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:57 smithi043 ceph-mon[41444]: osdmap e404: 8 total, 8 up, 8 in 2024-02-05T07:42:57.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:57 smithi050 ceph-mon[33492]: osdmap e404: 8 total, 8 up, 8 in 2024-02-05T07:42:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[41444]: pgmap v582: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[41444]: osdmap e405: 8 total, 8 up, 8 in 2024-02-05T07:42:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[29879]: pgmap v582: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[29879]: osdmap e405: 8 total, 8 up, 8 in 2024-02-05T07:42:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:58.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:58 smithi050 ceph-mon[33492]: pgmap v582: 260 pgs: 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:42:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:58 smithi050 ceph-mon[33492]: osdmap e405: 8 total, 8 up, 8 in 2024-02-05T07:42:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:42:58.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:42:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:59 smithi050 ceph-mon[33492]: osdmap e406: 8 total, 8 up, 8 in 2024-02-05T07:42:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:59.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:42:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:42:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:42:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[41444]: osdmap e406: 8 total, 8 up, 8 in 2024-02-05T07:42:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:42:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[29879]: osdmap e406: 8 total, 8 up, 8 in 2024-02-05T07:42:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:42:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:42:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:42:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:43:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:00 smithi043 ceph-mon[41444]: pgmap v585: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:43:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:00 smithi043 ceph-mon[41444]: osdmap e407: 8 total, 8 up, 8 in 2024-02-05T07:43:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:00 smithi043 ceph-mon[29879]: pgmap v585: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:43:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:00 smithi043 ceph-mon[29879]: osdmap e407: 8 total, 8 up, 8 in 2024-02-05T07:43:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:00 smithi050 ceph-mon[33492]: pgmap v585: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:43:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:00 smithi050 ceph-mon[33492]: osdmap e407: 8 total, 8 up, 8 in 2024-02-05T07:43:01.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:01 smithi050 ceph-mon[33492]: osdmap e408: 8 total, 8 up, 8 in 2024-02-05T07:43:01.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:43:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:43:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:01 smithi043 ceph-mon[29879]: osdmap e408: 8 total, 8 up, 8 in 2024-02-05T07:43:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:01 smithi043 ceph-mon[41444]: osdmap e408: 8 total, 8 up, 8 in 2024-02-05T07:43:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:02 smithi043 ceph-mon[29879]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:43:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:02 smithi043 ceph-mon[29879]: osdmap e409: 8 total, 8 up, 8 in 2024-02-05T07:43:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:02 smithi043 ceph-mon[41444]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:43:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:02 smithi043 ceph-mon[41444]: osdmap e409: 8 total, 8 up, 8 in 2024-02-05T07:43:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:02 smithi050 ceph-mon[33492]: pgmap v588: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:43:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:02 smithi050 ceph-mon[33492]: osdmap e409: 8 total, 8 up, 8 in 2024-02-05T07:43:03.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:03 smithi043 ceph-mon[41444]: osdmap e410: 8 total, 8 up, 8 in 2024-02-05T07:43:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:03 smithi043 ceph-mon[29879]: osdmap e410: 8 total, 8 up, 8 in 2024-02-05T07:43:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:03 smithi050 ceph-mon[33492]: osdmap e410: 8 total, 8 up, 8 in 2024-02-05T07:43:04.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:04 smithi043 ceph-mon[29879]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:43:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:04 smithi043 ceph-mon[41444]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:43:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:04 smithi050 ceph-mon[33492]: pgmap v591: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 5 op/s 2024-02-05T07:43:06.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:06 smithi043 ceph-mon[29879]: pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-05T07:43:06.365 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:06 smithi043 ceph-mon[41444]: pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-05T07:43:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:06 smithi050 ceph-mon[33492]: pgmap v592: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-05T07:43:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:08 smithi043 ceph-mon[41444]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-05T07:43:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:08 smithi043 ceph-mon[29879]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-05T07:43:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:08 smithi050 ceph-mon[33492]: pgmap v593: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2024-02-05T07:43:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:43:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:43:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:43:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:10 smithi043 ceph-mon[29879]: pgmap v594: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:10 smithi043 ceph-mon[41444]: pgmap v594: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:10 smithi050 ceph-mon[33492]: pgmap v594: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:11.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:43:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:43:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:12 smithi043 ceph-mon[29879]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 211 B/s wr, 2 op/s 2024-02-05T07:43:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:12 smithi043 ceph-mon[29879]: osdmap e411: 8 total, 8 up, 8 in 2024-02-05T07:43:12.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:12.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:12 smithi043 ceph-mon[41444]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 211 B/s wr, 2 op/s 2024-02-05T07:43:12.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:12 smithi043 ceph-mon[41444]: osdmap e411: 8 total, 8 up, 8 in 2024-02-05T07:43:12.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:12.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:12 smithi050 ceph-mon[33492]: pgmap v595: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 211 B/s wr, 2 op/s 2024-02-05T07:43:12.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:12 smithi050 ceph-mon[33492]: osdmap e411: 8 total, 8 up, 8 in 2024-02-05T07:43:12.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-65"}]: dispatch 2024-02-05T07:43:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-65"}]: dispatch 2024-02-05T07:43:13.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:13.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-65"}]: dispatch 2024-02-05T07:43:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-65"}]: dispatch 2024-02-05T07:43:13.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-65"}]: dispatch 2024-02-05T07:43:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-65"}]: dispatch 2024-02-05T07:43:13.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:14 smithi043 ceph-mon[29879]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-05T07:43:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:14 smithi043 ceph-mon[29879]: osdmap e412: 8 total, 8 up, 8 in 2024-02-05T07:43:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:14 smithi043 ceph-mon[41444]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-05T07:43:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:14 smithi043 ceph-mon[41444]: osdmap e412: 8 total, 8 up, 8 in 2024-02-05T07:43:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:14 smithi050 ceph-mon[33492]: pgmap v597: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 204 B/s wr, 2 op/s 2024-02-05T07:43:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:14 smithi050 ceph-mon[33492]: osdmap e412: 8 total, 8 up, 8 in 2024-02-05T07:43:14.449 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4ba0 2024-02-05T07:43:14.449 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4700 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4850 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4980 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4cf0 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b51a0 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b52d0 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b28c0 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b29f0 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b58a0 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5400 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5530 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b2b00 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5660 2024-02-05T07:43:14.450 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5790 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b59d0 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5e40 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5fb0 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4e20 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b4f70 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b65a0 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6100 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6230 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6360 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6490 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6b70 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b66d0 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6820 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6950 2024-02-05T07:43:14.451 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6cc0 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7150 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7280 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5ae0 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b5c10 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7850 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b73b0 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b74e0 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7610 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7740 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7e40 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b79a0 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7ab0 2024-02-05T07:43:14.452 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7be0 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7d10 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b83f0 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b7f50 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8080 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b81b0 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8540 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b89b0 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8ae0 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6df0 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b6f20 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9090 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8bf0 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8d20 2024-02-05T07:43:14.453 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8e70 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b91c0 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9650 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9780 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8650 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8760 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b8890 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b98b0 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9d40 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9e90 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b92f0 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9420 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ba460 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9fc0 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ba0f0 2024-02-05T07:43:14.454 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ba220 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ba350 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1baa30 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ba590 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ba6e0 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1ba830 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bab80 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb010 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb140 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b99e0 2024-02-05T07:43:14.455 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9af0 2024-02-05T07:43:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:15 smithi043 ceph-mon[41444]: osdmap e413: 8 total, 8 up, 8 in 2024-02-05T07:43:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:15.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:15 smithi043 ceph-mon[29879]: osdmap e413: 8 total, 8 up, 8 in 2024-02-05T07:43:15.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:15.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:15.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:15 smithi050 ceph-mon[33492]: osdmap e413: 8 total, 8 up, 8 in 2024-02-05T07:43:15.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:15.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:16.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:16 smithi050 ceph-mon[33492]: pgmap v600: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:16 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:16 smithi050 ceph-mon[33492]: osdmap e414: 8 total, 8 up, 8 in 2024-02-05T07:43:16.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[41444]: pgmap v600: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[41444]: osdmap e414: 8 total, 8 up, 8 in 2024-02-05T07:43:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[29879]: pgmap v600: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[29879]: osdmap e414: 8 total, 8 up, 8 in 2024-02-05T07:43:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:17 smithi050 ceph-mon[33492]: osdmap e415: 8 total, 8 up, 8 in 2024-02-05T07:43:17.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-67"}]: dispatch 2024-02-05T07:43:17.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-67"}]: dispatch 2024-02-05T07:43:17.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[29879]: osdmap e415: 8 total, 8 up, 8 in 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-67"}]: dispatch 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-67"}]: dispatch 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[41444]: osdmap e415: 8 total, 8 up, 8 in 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-67"}]: dispatch 2024-02-05T07:43:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-67"}]: dispatch 2024-02-05T07:43:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:18 smithi043 ceph-mon[29879]: pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:18 smithi043 ceph-mon[29879]: osdmap e416: 8 total, 8 up, 8 in 2024-02-05T07:43:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:18 smithi043 ceph-mon[41444]: pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:18 smithi043 ceph-mon[41444]: osdmap e416: 8 total, 8 up, 8 in 2024-02-05T07:43:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:18 smithi050 ceph-mon[33492]: pgmap v603: 292 pgs: 292 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:18 smithi050 ceph-mon[33492]: osdmap e416: 8 total, 8 up, 8 in 2024-02-05T07:43:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:43:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:43:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[29879]: osdmap e417: 8 total, 8 up, 8 in 2024-02-05T07:43:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[29879]: osdmap e418: 8 total, 8 up, 8 in 2024-02-05T07:43:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[41444]: osdmap e417: 8 total, 8 up, 8 in 2024-02-05T07:43:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:19.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:19 smithi043 ceph-mon[41444]: osdmap e418: 8 total, 8 up, 8 in 2024-02-05T07:43:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:19 smithi050 ceph-mon[33492]: osdmap e417: 8 total, 8 up, 8 in 2024-02-05T07:43:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:19 smithi050 ceph-mon[33492]: osdmap e418: 8 total, 8 up, 8 in 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[29879]: pgmap v606: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-69"}]: dispatch 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-69"}]: dispatch 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[29879]: osdmap e419: 8 total, 8 up, 8 in 2024-02-05T07:43:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[41444]: pgmap v606: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:43:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-69"}]: dispatch 2024-02-05T07:43:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-69"}]: dispatch 2024-02-05T07:43:20.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:20 smithi043 ceph-mon[41444]: osdmap e419: 8 total, 8 up, 8 in 2024-02-05T07:43:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:20 smithi050 ceph-mon[33492]: pgmap v606: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:43:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-69"}]: dispatch 2024-02-05T07:43:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-69"}]: dispatch 2024-02-05T07:43:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:20 smithi050 ceph-mon[33492]: osdmap e419: 8 total, 8 up, 8 in 2024-02-05T07:43:21.363 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:43:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[29879]: pgmap v609: 260 pgs: 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[29879]: osdmap e420: 8 total, 8 up, 8 in 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[41444]: pgmap v609: 260 pgs: 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T07:43:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[41444]: osdmap e420: 8 total, 8 up, 8 in 2024-02-05T07:43:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:21 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:43:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:21 smithi050 ceph-mon[33492]: pgmap v609: 260 pgs: 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-05T07:43:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:21 smithi050 ceph-mon[33492]: osdmap e420: 8 total, 8 up, 8 in 2024-02-05T07:43:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: osdmap e421: 8 total, 8 up, 8 in 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:43:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: osdmap e421: 8 total, 8 up, 8 in 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:43:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:43:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:43:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:43:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:43:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:22.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: osdmap e421: 8 total, 8 up, 8 in 2024-02-05T07:43:22.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:22.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:43:22.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:43:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:24 smithi043 ceph-mon[29879]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:43:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:43:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:24 smithi043 ceph-mon[29879]: osdmap e422: 8 total, 8 up, 8 in 2024-02-05T07:43:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:24 smithi043 ceph-mon[41444]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:43:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:43:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:24 smithi043 ceph-mon[41444]: osdmap e422: 8 total, 8 up, 8 in 2024-02-05T07:43:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:24 smithi050 ceph-mon[33492]: pgmap v612: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:43:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:43:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:24 smithi050 ceph-mon[33492]: osdmap e422: 8 total, 8 up, 8 in 2024-02-05T07:43:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:25 smithi043 ceph-mon[29879]: osdmap e423: 8 total, 8 up, 8 in 2024-02-05T07:43:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:25 smithi043 ceph-mon[41444]: osdmap e423: 8 total, 8 up, 8 in 2024-02-05T07:43:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:25 smithi050 ceph-mon[33492]: osdmap e423: 8 total, 8 up, 8 in 2024-02-05T07:43:26.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:26 smithi050 ceph-mon[33492]: pgmap v615: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:26.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:26 smithi043 ceph-mon[29879]: pgmap v615: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:26 smithi043 ceph-mon[41444]: pgmap v615: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:28 smithi043 ceph-mon[29879]: pgmap v616: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-05T07:43:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:28 smithi043 ceph-mon[41444]: pgmap v616: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-05T07:43:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:28 smithi050 ceph-mon[33492]: pgmap v616: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.3 KiB/s wr, 5 op/s 2024-02-05T07:43:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:43:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:43:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:43:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:30 smithi043 ceph-mon[41444]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 918 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-05T07:43:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:30 smithi043 ceph-mon[29879]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 918 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-05T07:43:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:30 smithi050 ceph-mon[33492]: pgmap v617: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 918 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-05T07:43:31.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:43:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:43:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:32 smithi043 ceph-mon[41444]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-02-05T07:43:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:32 smithi043 ceph-mon[29879]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-02-05T07:43:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:32 smithi050 ceph-mon[33492]: pgmap v618: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-02-05T07:43:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:34.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:34 smithi043 ceph-mon[29879]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-05T07:43:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:34 smithi043 ceph-mon[41444]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-05T07:43:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:34 smithi050 ceph-mon[33492]: pgmap v619: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-05T07:43:36.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:36 smithi043 ceph-mon[29879]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-05T07:43:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:36 smithi043 ceph-mon[41444]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-05T07:43:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:36 smithi050 ceph-mon[33492]: pgmap v620: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-05T07:43:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:38 smithi043 ceph-mon[29879]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-05T07:43:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:38 smithi043 ceph-mon[41444]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-05T07:43:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:38 smithi050 ceph-mon[33492]: pgmap v621: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-02-05T07:43:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:43:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:43:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:43:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:40 smithi043 ceph-mon[29879]: pgmap v622: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:40 smithi043 ceph-mon[41444]: pgmap v622: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:40 smithi050 ceph-mon[33492]: pgmap v622: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]: dispatch 2024-02-05T07:43:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]: dispatch 2024-02-05T07:43:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]: dispatch 2024-02-05T07:43:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]: dispatch 2024-02-05T07:43:41.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]: dispatch 2024-02-05T07:43:41.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:43:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:43:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:41 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]: dispatch 2024-02-05T07:43:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:41 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]: dispatch 2024-02-05T07:43:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]: dispatch 2024-02-05T07:43:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:41 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]: dispatch 2024-02-05T07:43:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]: dispatch 2024-02-05T07:43:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]: dispatch 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[29879]: pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]': finished 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]': finished 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]': finished 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[29879]: osdmap e424: 8 total, 8 up, 8 in 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[41444]: pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]': finished 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]': finished 2024-02-05T07:43:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]': finished 2024-02-05T07:43:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:42 smithi043 ceph-mon[41444]: osdmap e424: 8 total, 8 up, 8 in 2024-02-05T07:43:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:42 smithi050 ceph-mon[33492]: pgmap v623: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:43:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:42 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.7", "id": [1, 2]}]': finished 2024-02-05T07:43:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:42 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.d", "id": [6, 2]}]': finished 2024-02-05T07:43:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:42 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "300.19", "id": [7, 3]}]': finished 2024-02-05T07:43:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:42 smithi050 ceph-mon[33492]: osdmap e424: 8 total, 8 up, 8 in 2024-02-05T07:43:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:43 smithi043 ceph-mon[29879]: osdmap e425: 8 total, 8 up, 8 in 2024-02-05T07:43:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:43 smithi043 ceph-mon[41444]: osdmap e425: 8 total, 8 up, 8 in 2024-02-05T07:43:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:43 smithi050 ceph-mon[33492]: osdmap e425: 8 total, 8 up, 8 in 2024-02-05T07:43:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:44 smithi043 ceph-mon[29879]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-02-05T07:43:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:44 smithi043 ceph-mon[41444]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-02-05T07:43:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:44 smithi050 ceph-mon[33492]: pgmap v626: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 383 B/s wr, 3 op/s 2024-02-05T07:43:45.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:45.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:45.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-71"}]: dispatch 2024-02-05T07:43:45.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-71"}]: dispatch 2024-02-05T07:43:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-71"}]: dispatch 2024-02-05T07:43:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-71"}]: dispatch 2024-02-05T07:43:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:43:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-71"}]: dispatch 2024-02-05T07:43:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-71"}]: dispatch 2024-02-05T07:43:46.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:46 smithi043 ceph-mon[29879]: pgmap v627: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:43:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:46 smithi043 ceph-mon[29879]: osdmap e426: 8 total, 8 up, 8 in 2024-02-05T07:43:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:46 smithi043 ceph-mon[41444]: pgmap v627: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:43:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:46 smithi043 ceph-mon[41444]: osdmap e426: 8 total, 8 up, 8 in 2024-02-05T07:43:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:46 smithi050 ceph-mon[33492]: pgmap v627: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:43:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:46 smithi050 ceph-mon[33492]: osdmap e426: 8 total, 8 up, 8 in 2024-02-05T07:43:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[29879]: osdmap e427: 8 total, 8 up, 8 in 2024-02-05T07:43:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[29879]: osdmap e428: 8 total, 8 up, 8 in 2024-02-05T07:43:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[41444]: osdmap e427: 8 total, 8 up, 8 in 2024-02-05T07:43:47.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:47.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:47.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:47.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:47 smithi043 ceph-mon[41444]: osdmap e428: 8 total, 8 up, 8 in 2024-02-05T07:43:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:47 smithi050 ceph-mon[33492]: osdmap e427: 8 total, 8 up, 8 in 2024-02-05T07:43:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:43:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:43:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:47 smithi050 ceph-mon[33492]: osdmap e428: 8 total, 8 up, 8 in 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[29879]: pgmap v630: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[29879]: osdmap e429: 8 total, 8 up, 8 in 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[41444]: pgmap v630: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:48 smithi043 ceph-mon[41444]: osdmap e429: 8 total, 8 up, 8 in 2024-02-05T07:43:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:48 smithi050 ceph-mon[33492]: pgmap v630: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:43:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:48 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:48 smithi050 ceph-mon[33492]: osdmap e429: 8 total, 8 up, 8 in 2024-02-05T07:43:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:43:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:43:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:43:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:50 smithi043 ceph-mon[29879]: pgmap v633: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:50 smithi043 ceph-mon[29879]: osdmap e430: 8 total, 8 up, 8 in 2024-02-05T07:43:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:50 smithi043 ceph-mon[41444]: pgmap v633: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:50 smithi043 ceph-mon[41444]: osdmap e430: 8 total, 8 up, 8 in 2024-02-05T07:43:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:50 smithi050 ceph-mon[33492]: pgmap v633: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:43:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:50 smithi050 ceph-mon[33492]: osdmap e430: 8 total, 8 up, 8 in 2024-02-05T07:43:51.576 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:43:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:43:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:52 smithi043 ceph-mon[29879]: pgmap v635: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-05T07:43:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:52 smithi043 ceph-mon[41444]: pgmap v635: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-05T07:43:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:52 smithi050 ceph-mon[33492]: pgmap v635: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-05T07:43:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:43:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:53 smithi043 ceph-mon[29879]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-05T07:43:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:53 smithi043 ceph-mon[41444]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-05T07:43:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:53 smithi050 ceph-mon[33492]: pgmap v636: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-05T07:43:56.275 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:55 smithi050 ceph-mon[33492]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 938 B/s wr, 3 op/s 2024-02-05T07:43:56.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:55 smithi043 ceph-mon[29879]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 938 B/s wr, 3 op/s 2024-02-05T07:43:56.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:55 smithi043 ceph-mon[41444]: pgmap v637: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 938 B/s wr, 3 op/s 2024-02-05T07:43:57.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:56 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:57.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:56 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:57.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:56 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:43:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:58 smithi043 ceph-mon[29879]: pgmap v638: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 831 B/s wr, 3 op/s 2024-02-05T07:43:58.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:43:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:58 smithi043 ceph-mon[41444]: pgmap v638: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 831 B/s wr, 3 op/s 2024-02-05T07:43:58.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:43:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:58 smithi050 ceph-mon[33492]: pgmap v638: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 831 B/s wr, 3 op/s 2024-02-05T07:43:58.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:43:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:43:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:43:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:43:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:44:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:00 smithi050 ceph-mon[33492]: pgmap v639: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-05T07:44:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-73"}]: dispatch 2024-02-05T07:44:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-73"}]: dispatch 2024-02-05T07:44:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[29879]: pgmap v639: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-05T07:44:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-73"}]: dispatch 2024-02-05T07:44:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-73"}]: dispatch 2024-02-05T07:44:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[41444]: pgmap v639: 292 pgs: 292 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-05T07:44:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-73"}]: dispatch 2024-02-05T07:44:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-73"}]: dispatch 2024-02-05T07:44:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:01 smithi050 ceph-mon[33492]: osdmap e431: 8 total, 8 up, 8 in 2024-02-05T07:44:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:01 smithi050 ceph-mon[33492]: pgmap v641: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:44:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:01 smithi050 ceph-mon[33492]: osdmap e432: 8 total, 8 up, 8 in 2024-02-05T07:44:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:01 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:01.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:44:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[29879]: osdmap e431: 8 total, 8 up, 8 in 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[29879]: pgmap v641: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[29879]: osdmap e432: 8 total, 8 up, 8 in 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[41444]: osdmap e431: 8 total, 8 up, 8 in 2024-02-05T07:44:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[41444]: pgmap v641: 260 pgs: 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:44:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[41444]: osdmap e432: 8 total, 8 up, 8 in 2024-02-05T07:44:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:01 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:03.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:03 smithi050 ceph-mon[33492]: osdmap e433: 8 total, 8 up, 8 in 2024-02-05T07:44:03.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:03 smithi043 ceph-mon[41444]: osdmap e433: 8 total, 8 up, 8 in 2024-02-05T07:44:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:03 smithi043 ceph-mon[29879]: osdmap e433: 8 total, 8 up, 8 in 2024-02-05T07:44:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:04.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:04 smithi043 ceph-mon[41444]: pgmap v644: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:04 smithi043 ceph-mon[41444]: osdmap e434: 8 total, 8 up, 8 in 2024-02-05T07:44:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:04 smithi043 ceph-mon[29879]: pgmap v644: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:04 smithi043 ceph-mon[29879]: osdmap e434: 8 total, 8 up, 8 in 2024-02-05T07:44:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:04 smithi050 ceph-mon[33492]: pgmap v644: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:04 smithi050 ceph-mon[33492]: osdmap e434: 8 total, 8 up, 8 in 2024-02-05T07:44:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[29879]: osdmap e435: 8 total, 8 up, 8 in 2024-02-05T07:44:05.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-75"}]: dispatch 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-75"}]: dispatch 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[41444]: osdmap e435: 8 total, 8 up, 8 in 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-75"}]: dispatch 2024-02-05T07:44:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-75"}]: dispatch 2024-02-05T07:44:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:05 smithi050 ceph-mon[33492]: osdmap e435: 8 total, 8 up, 8 in 2024-02-05T07:44:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-75"}]: dispatch 2024-02-05T07:44:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-75"}]: dispatch 2024-02-05T07:44:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:06 smithi043 ceph-mon[29879]: pgmap v647: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:06 smithi043 ceph-mon[29879]: osdmap e436: 8 total, 8 up, 8 in 2024-02-05T07:44:06.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:06 smithi043 ceph-mon[41444]: pgmap v647: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:06.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:06 smithi043 ceph-mon[41444]: osdmap e436: 8 total, 8 up, 8 in 2024-02-05T07:44:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:06 smithi050 ceph-mon[33492]: pgmap v647: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:06 smithi050 ceph-mon[33492]: osdmap e436: 8 total, 8 up, 8 in 2024-02-05T07:44:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[29879]: osdmap e437: 8 total, 8 up, 8 in 2024-02-05T07:44:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[29879]: osdmap e438: 8 total, 8 up, 8 in 2024-02-05T07:44:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[41444]: osdmap e437: 8 total, 8 up, 8 in 2024-02-05T07:44:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:07 smithi043 ceph-mon[41444]: osdmap e438: 8 total, 8 up, 8 in 2024-02-05T07:44:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:07 smithi050 ceph-mon[33492]: osdmap e437: 8 total, 8 up, 8 in 2024-02-05T07:44:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:07 smithi050 ceph-mon[33492]: osdmap e438: 8 total, 8 up, 8 in 2024-02-05T07:44:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[41444]: pgmap v650: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T07:44:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[41444]: osdmap e439: 8 total, 8 up, 8 in 2024-02-05T07:44:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[29879]: pgmap v650: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T07:44:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:08 smithi043 ceph-mon[29879]: osdmap e439: 8 total, 8 up, 8 in 2024-02-05T07:44:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:08 smithi050 ceph-mon[33492]: pgmap v650: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T07:44:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:08 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:08 smithi050 ceph-mon[33492]: osdmap e439: 8 total, 8 up, 8 in 2024-02-05T07:44:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:44:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-77"}]: dispatch 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-77"}]: dispatch 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-77"}]: dispatch 2024-02-05T07:44:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-77"}]: dispatch 2024-02-05T07:44:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-77"}]: dispatch 2024-02-05T07:44:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-77"}]: dispatch 2024-02-05T07:44:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:10 smithi043 ceph-mon[29879]: pgmap v653: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T07:44:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:10 smithi043 ceph-mon[29879]: osdmap e440: 8 total, 8 up, 8 in 2024-02-05T07:44:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:10 smithi043 ceph-mon[41444]: pgmap v653: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T07:44:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:10 smithi043 ceph-mon[41444]: osdmap e440: 8 total, 8 up, 8 in 2024-02-05T07:44:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:10 smithi050 ceph-mon[33492]: pgmap v653: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 108 KiB/s rd, 767 B/s wr, 181 op/s 2024-02-05T07:44:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:10 smithi050 ceph-mon[33492]: osdmap e440: 8 total, 8 up, 8 in 2024-02-05T07:44:11.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:11 smithi043 ceph-mon[29879]: osdmap e441: 8 total, 8 up, 8 in 2024-02-05T07:44:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:11 smithi043 ceph-mon[41444]: osdmap e441: 8 total, 8 up, 8 in 2024-02-05T07:44:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:11.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:44:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:44:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:11 smithi050 ceph-mon[33492]: osdmap e441: 8 total, 8 up, 8 in 2024-02-05T07:44:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[41444]: pgmap v656: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[41444]: osdmap e442: 8 total, 8 up, 8 in 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[29879]: pgmap v656: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[29879]: osdmap e442: 8 total, 8 up, 8 in 2024-02-05T07:44:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:12 smithi050 ceph-mon[33492]: pgmap v656: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:44:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:12 smithi050 ceph-mon[33492]: osdmap e442: 8 total, 8 up, 8 in 2024-02-05T07:44:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:13 smithi043 ceph-mon[41444]: osdmap e443: 8 total, 8 up, 8 in 2024-02-05T07:44:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:13 smithi043 ceph-mon[29879]: osdmap e443: 8 total, 8 up, 8 in 2024-02-05T07:44:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:13 smithi050 ceph-mon[33492]: osdmap e443: 8 total, 8 up, 8 in 2024-02-05T07:44:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[41444]: pgmap v659: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[41444]: osdmap e444: 8 total, 8 up, 8 in 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.15"}]: dispatch 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.15"}]: dispatch 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[41444]: 304.15 deep-scrub starts 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[41444]: 304.15 deep-scrub ok 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[29879]: pgmap v659: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[29879]: osdmap e444: 8 total, 8 up, 8 in 2024-02-05T07:44:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.15"}]: dispatch 2024-02-05T07:44:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.15"}]: dispatch 2024-02-05T07:44:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[29879]: 304.15 deep-scrub starts 2024-02-05T07:44:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:14 smithi043 ceph-mon[29879]: 304.15 deep-scrub ok 2024-02-05T07:44:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:14 smithi050 ceph-mon[33492]: pgmap v659: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:44:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:14 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:14 smithi050 ceph-mon[33492]: osdmap e444: 8 total, 8 up, 8 in 2024-02-05T07:44:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.15"}]: dispatch 2024-02-05T07:44:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:14 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "304.15"}]: dispatch 2024-02-05T07:44:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:14 smithi050 ceph-mon[33492]: 304.15 deep-scrub starts 2024-02-05T07:44:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:14 smithi050 ceph-mon[33492]: 304.15 deep-scrub ok 2024-02-05T07:44:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:16 smithi043 ceph-mon[41444]: pgmap v661: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 437 B/s wr, 2 op/s 2024-02-05T07:44:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:16 smithi043 ceph-mon[29879]: pgmap v661: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 437 B/s wr, 2 op/s 2024-02-05T07:44:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:16 smithi050 ceph-mon[33492]: pgmap v661: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 658 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 437 B/s wr, 2 op/s 2024-02-05T07:44:18.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:18 smithi043 ceph-mon[41444]: pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-02-05T07:44:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:18 smithi043 ceph-mon[29879]: pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-02-05T07:44:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:18 smithi050 ceph-mon[33492]: pgmap v662: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-02-05T07:44:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:44:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:44:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:44:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:20 smithi043 ceph-mon[29879]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 666 B/s rd, 666 B/s wr, 1 op/s 2024-02-05T07:44:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:20 smithi043 ceph-mon[41444]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 666 B/s rd, 666 B/s wr, 1 op/s 2024-02-05T07:44:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:20 smithi050 ceph-mon[33492]: pgmap v663: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 666 B/s rd, 666 B/s wr, 1 op/s 2024-02-05T07:44:21.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:44:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:44:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[29879]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 590 B/s wr, 2 op/s 2024-02-05T07:44:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:44:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:44:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:44:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[41444]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 590 B/s wr, 2 op/s 2024-02-05T07:44:22.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:22.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:22.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:44:22.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:44:22.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:22 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:44:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:22 smithi050 ceph-mon[33492]: pgmap v664: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 590 B/s wr, 2 op/s 2024-02-05T07:44:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:22 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:22 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:44:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:22 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:44:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:22 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:44:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:22 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:44:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:24 smithi043 ceph-mon[29879]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:44:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:24 smithi043 ceph-mon[41444]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:44:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:24 smithi050 ceph-mon[33492]: pgmap v665: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:44:26.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:26 smithi043 ceph-mon[29879]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 877 B/s rd, 438 B/s wr, 1 op/s 2024-02-05T07:44:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:26 smithi043 ceph-mon[41444]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 877 B/s rd, 438 B/s wr, 1 op/s 2024-02-05T07:44:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:26 smithi050 ceph-mon[33492]: pgmap v666: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 877 B/s rd, 438 B/s wr, 1 op/s 2024-02-05T07:44:28.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:28 smithi043 ceph-mon[29879]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-02-05T07:44:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:28 smithi043 ceph-mon[41444]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-02-05T07:44:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:28 smithi050 ceph-mon[33492]: pgmap v667: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 2 op/s 2024-02-05T07:44:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:44:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:44:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:44:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:30 smithi043 ceph-mon[41444]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:44:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:30 smithi043 ceph-mon[29879]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:44:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:30 smithi050 ceph-mon[33492]: pgmap v668: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:44:31.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:44:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:44:32.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:32 smithi043 ceph-mon[29879]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:44:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:32 smithi043 ceph-mon[41444]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:44:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:32 smithi050 ceph-mon[33492]: pgmap v669: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:44:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:33.563 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 07:44:33 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0[45879]: 2024-02-05T07:44:33.176+0000 7f82b952b700 -1 snap_mapper.add_oid found existing snaps mapped on 304:ad639276:test-rados-api-smithi043-81993-80::foo:2, removing 2024-02-05T07:44:33.563 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 07:44:33 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2[58158]: 2024-02-05T07:44:33.176+0000 7f205b2be700 -1 snap_mapper.add_oid found existing snaps mapped on 304:ad639276:test-rados-api-smithi043-81993-80::foo:2, removing 2024-02-05T07:44:33.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 07:44:33 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T07:44:33.175+0000 7f9271ac6700 -1 snap_mapper.add_oid found existing snaps mapped on 304:ad639276:test-rados-api-smithi043-81993-80::foo:2, removing 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[29879]: pgmap v670: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-79"}]: dispatch 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-79"}]: dispatch 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[41444]: pgmap v670: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-79"}]: dispatch 2024-02-05T07:44:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-79"}]: dispatch 2024-02-05T07:44:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:34 smithi050 ceph-mon[33492]: pgmap v670: 292 pgs: 292 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:44:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-79"}]: dispatch 2024-02-05T07:44:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-79"}]: dispatch 2024-02-05T07:44:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:35 smithi043 ceph-mon[29879]: osdmap e445: 8 total, 8 up, 8 in 2024-02-05T07:44:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:35 smithi043 ceph-mon[41444]: osdmap e445: 8 total, 8 up, 8 in 2024-02-05T07:44:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:35 smithi050 ceph-mon[33492]: osdmap e445: 8 total, 8 up, 8 in 2024-02-05T07:44:36.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[29879]: pgmap v672: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:44:36.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[29879]: osdmap e446: 8 total, 8 up, 8 in 2024-02-05T07:44:36.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:36.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:36.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[41444]: pgmap v672: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:44:36.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[41444]: osdmap e446: 8 total, 8 up, 8 in 2024-02-05T07:44:36.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:36.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:36.466 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1b9c20 2024-02-05T07:44:36.466 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb270 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb6e0 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb810 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bacb0 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bae00 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bbdc0 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb920 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bba50 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bbb80 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bbcb0 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc3b0 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bbf10 2024-02-05T07:44:36.467 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc040 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc150 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc2a0 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc980 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc4e0 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc630 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bc780 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bcab0 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bcf40 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd070 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb380 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bb4b0 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd640 2024-02-05T07:44:36.468 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd1a0 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd2d0 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd400 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd530 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bdc10 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd770 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd8c0 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bd9f0 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bdd40 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1be1d0 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1be300 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bdb00 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1be790 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1be8a0 2024-02-05T07:44:36.469 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bea10 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bde50 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bdfa0 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bcbc0 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bccf0 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bce20 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1beb60 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1beff0 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf140 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1be410 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1be560 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf710 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf270 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf3a0 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf4d0 2024-02-05T07:44:36.470 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf600 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bfce0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf840 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bf990 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bfaa0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bfbb0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c02b0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bfe10 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bff60 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c00b0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0400 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c08e0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0a10 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bec90 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1bedc0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0fe0 2024-02-05T07:44:36.471 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0b40 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0c70 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0da0 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1110 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c15f0 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1a80 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1bb0 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1240 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1370 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c14a0 2024-02-05T07:44:36.472 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0510 2024-02-05T07:44:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:36 smithi050 ceph-mon[33492]: pgmap v672: 260 pgs: 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:44:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:36 smithi050 ceph-mon[33492]: osdmap e446: 8 total, 8 up, 8 in 2024-02-05T07:44:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[29879]: osdmap e447: 8 total, 8 up, 8 in 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[41444]: osdmap e447: 8 total, 8 up, 8 in 2024-02-05T07:44:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:37.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:37.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:37 smithi050 ceph-mon[33492]: osdmap e447: 8 total, 8 up, 8 in 2024-02-05T07:44:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[29879]: pgmap v675: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[29879]: osdmap e448: 8 total, 8 up, 8 in 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[41444]: pgmap v675: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:38.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[41444]: osdmap e448: 8 total, 8 up, 8 in 2024-02-05T07:44:38.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:38 smithi050 ceph-mon[33492]: pgmap v675: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-05T07:44:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:38 smithi050 ceph-mon[33492]: osdmap e448: 8 total, 8 up, 8 in 2024-02-05T07:44:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:44:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:44:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[29879]: osdmap e449: 8 total, 8 up, 8 in 2024-02-05T07:44:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[29879]: osdmap e450: 8 total, 8 up, 8 in 2024-02-05T07:44:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[41444]: osdmap e449: 8 total, 8 up, 8 in 2024-02-05T07:44:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:39 smithi043 ceph-mon[41444]: osdmap e450: 8 total, 8 up, 8 in 2024-02-05T07:44:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:39 smithi050 ceph-mon[33492]: osdmap e449: 8 total, 8 up, 8 in 2024-02-05T07:44:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:39 smithi050 ceph-mon[33492]: osdmap e450: 8 total, 8 up, 8 in 2024-02-05T07:44:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[29879]: pgmap v678: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:44:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[29879]: osdmap e451: 8 total, 8 up, 8 in 2024-02-05T07:44:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[41444]: pgmap v678: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:44:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:40 smithi043 ceph-mon[41444]: osdmap e451: 8 total, 8 up, 8 in 2024-02-05T07:44:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:40 smithi050 ceph-mon[33492]: pgmap v678: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:44:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:40 smithi050 ceph-mon[33492]: osdmap e451: 8 total, 8 up, 8 in 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[29879]: pgmap v681: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[41444]: pgmap v681: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-05T07:44:41.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:41 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:41.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:44:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:44:41.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T07:44:41.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-05T07:44:41.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:41 smithi050 ceph-mon[33492]: pgmap v681: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-05T07:44:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:41 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-05T07:44:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[29879]: osdmap e452: 8 total, 8 up, 8 in 2024-02-05T07:44:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T07:44:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T07:44:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-05T07:44:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[41444]: osdmap e452: 8 total, 8 up, 8 in 2024-02-05T07:44:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T07:44:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T07:44:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-05T07:44:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:42 smithi050 ceph-mon[33492]: osdmap e452: 8 total, 8 up, 8 in 2024-02-05T07:44:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T07:44:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-05T07:44:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-05T07:44:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[29879]: osdmap e453: 8 total, 8 up, 8 in 2024-02-05T07:44:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[29879]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-05T07:44:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-05T07:44:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[41444]: osdmap e453: 8 total, 8 up, 8 in 2024-02-05T07:44:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:43 smithi043 ceph-mon[41444]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-05T07:44:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-05T07:44:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:43 smithi050 ceph-mon[33492]: osdmap e453: 8 total, 8 up, 8 in 2024-02-05T07:44:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:43 smithi050 ceph-mon[33492]: pgmap v684: 292 pgs: 292 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 5.2 KiB/s wr, 8 op/s 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[29879]: osdmap e454: 8 total, 8 up, 8 in 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-81"}]: dispatch 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-81"}]: dispatch 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[41444]: osdmap e454: 8 total, 8 up, 8 in 2024-02-05T07:44:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-81"}]: dispatch 2024-02-05T07:44:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-81"}]: dispatch 2024-02-05T07:44:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:44 smithi050 ceph-mon[33492]: osdmap e454: 8 total, 8 up, 8 in 2024-02-05T07:44:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-81"}]: dispatch 2024-02-05T07:44:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-81"}]: dispatch 2024-02-05T07:44:45.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:45 smithi043 ceph-mon[29879]: osdmap e455: 8 total, 8 up, 8 in 2024-02-05T07:44:45.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:45 smithi043 ceph-mon[29879]: pgmap v687: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:45 smithi043 ceph-mon[41444]: osdmap e455: 8 total, 8 up, 8 in 2024-02-05T07:44:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:45 smithi043 ceph-mon[41444]: pgmap v687: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:45 smithi050 ceph-mon[33492]: osdmap e455: 8 total, 8 up, 8 in 2024-02-05T07:44:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:45 smithi050 ceph-mon[33492]: pgmap v687: 260 pgs: 260 active+clean; 8.4 MiB data, 643 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:46 smithi043 ceph-mon[29879]: osdmap e456: 8 total, 8 up, 8 in 2024-02-05T07:44:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:46 smithi043 ceph-mon[41444]: osdmap e456: 8 total, 8 up, 8 in 2024-02-05T07:44:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:46 smithi050 ceph-mon[33492]: osdmap e456: 8 total, 8 up, 8 in 2024-02-05T07:44:46.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:46.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[41444]: osdmap e457: 8 total, 8 up, 8 in 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[41444]: pgmap v690: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[29879]: osdmap e457: 8 total, 8 up, 8 in 2024-02-05T07:44:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:47.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:47 smithi043 ceph-mon[29879]: pgmap v690: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-02-05T07:44:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:47 smithi050 ceph-mon[33492]: osdmap e457: 8 total, 8 up, 8 in 2024-02-05T07:44:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:47 smithi050 ceph-mon[33492]: pgmap v690: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 7.0 KiB/s wr, 32 op/s 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[41444]: osdmap e458: 8 total, 8 up, 8 in 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[29879]: osdmap e458: 8 total, 8 up, 8 in 2024-02-05T07:44:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:48 smithi050 ceph-mon[33492]: osdmap e458: 8 total, 8 up, 8 in 2024-02-05T07:44:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:44:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:44:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[41444]: osdmap e459: 8 total, 8 up, 8 in 2024-02-05T07:44:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[41444]: pgmap v693: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 9.0 KiB/s wr, 32 op/s 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[41444]: osdmap e460: 8 total, 8 up, 8 in 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[29879]: osdmap e459: 8 total, 8 up, 8 in 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[29879]: pgmap v693: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 9.0 KiB/s wr, 32 op/s 2024-02-05T07:44:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:49.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:49 smithi043 ceph-mon[29879]: osdmap e460: 8 total, 8 up, 8 in 2024-02-05T07:44:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:49 smithi050 ceph-mon[33492]: osdmap e459: 8 total, 8 up, 8 in 2024-02-05T07:44:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:49 smithi050 ceph-mon[33492]: pgmap v693: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 9.5 KiB/s rd, 9.0 KiB/s wr, 32 op/s 2024-02-05T07:44:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:49.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:49 smithi050 ceph-mon[33492]: osdmap e460: 8 total, 8 up, 8 in 2024-02-05T07:44:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[41444]: osdmap e461: 8 total, 8 up, 8 in 2024-02-05T07:44:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:50.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:50 smithi043 ceph-mon[29879]: osdmap e461: 8 total, 8 up, 8 in 2024-02-05T07:44:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:50 smithi050 ceph-mon[33492]: osdmap e461: 8 total, 8 up, 8 in 2024-02-05T07:44:51.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:44:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:44:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:52 smithi050 ceph-mon[33492]: pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:44:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:52 smithi050 ceph-mon[33492]: osdmap e462: 8 total, 8 up, 8 in 2024-02-05T07:44:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:52 smithi043 ceph-mon[29879]: pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:44:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:52 smithi043 ceph-mon[29879]: osdmap e462: 8 total, 8 up, 8 in 2024-02-05T07:44:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:52 smithi043 ceph-mon[41444]: pgmap v696: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:44:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:44:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:52 smithi043 ceph-mon[41444]: osdmap e462: 8 total, 8 up, 8 in 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[29879]: osdmap e463: 8 total, 8 up, 8 in 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-83"}]: dispatch 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-83"}]: dispatch 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[29879]: pgmap v699: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[41444]: osdmap e463: 8 total, 8 up, 8 in 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-83"}]: dispatch 2024-02-05T07:44:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-83"}]: dispatch 2024-02-05T07:44:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:53 smithi043 ceph-mon[41444]: pgmap v699: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:44:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:53 smithi050 ceph-mon[33492]: osdmap e463: 8 total, 8 up, 8 in 2024-02-05T07:44:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-83"}]: dispatch 2024-02-05T07:44:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-83"}]: dispatch 2024-02-05T07:44:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:53 smithi050 ceph-mon[33492]: pgmap v699: 292 pgs: 292 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:44:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:54 smithi043 ceph-mon[29879]: osdmap e464: 8 total, 8 up, 8 in 2024-02-05T07:44:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:54 smithi043 ceph-mon[41444]: osdmap e464: 8 total, 8 up, 8 in 2024-02-05T07:44:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:54 smithi050 ceph-mon[33492]: osdmap e464: 8 total, 8 up, 8 in 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[29879]: osdmap e465: 8 total, 8 up, 8 in 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[29879]: pgmap v702: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[41444]: osdmap e465: 8 total, 8 up, 8 in 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:55 smithi043 ceph-mon[41444]: pgmap v702: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:55 smithi050 ceph-mon[33492]: osdmap e465: 8 total, 8 up, 8 in 2024-02-05T07:44:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:44:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:55 smithi050 ceph-mon[33492]: pgmap v702: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[29879]: osdmap e466: 8 total, 8 up, 8 in 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[41444]: osdmap e466: 8 total, 8 up, 8 in 2024-02-05T07:44:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:56 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:44:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:44:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:56 smithi050 ceph-mon[33492]: osdmap e466: 8 total, 8 up, 8 in 2024-02-05T07:44:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:44:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:44:57.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[29879]: osdmap e467: 8 total, 8 up, 8 in 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[29879]: pgmap v705: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[41444]: osdmap e467: 8 total, 8 up, 8 in 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:57 smithi043 ceph-mon[41444]: pgmap v705: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-02-05T07:44:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:44:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:57 smithi050 ceph-mon[33492]: osdmap e467: 8 total, 8 up, 8 in 2024-02-05T07:44:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:44:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:57 smithi050 ceph-mon[33492]: pgmap v705: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: osdmap e468: 8 total, 8 up, 8 in 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: osdmap e469: 8 total, 8 up, 8 in 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: osdmap e468: 8 total, 8 up, 8 in 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: osdmap e469: 8 total, 8 up, 8 in 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_tier","val": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:44:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: osdmap e468: 8 total, 8 up, 8 in 2024-02-05T07:44:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:44:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:44:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:44:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: osdmap e469: 8 total, 8 up, 8 in 2024-02-05T07:44:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:44:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:44:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:44:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:44:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:59 smithi043 ceph-mon[29879]: pgmap v708: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-05T07:44:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:44:59 smithi043 ceph-mon[29879]: osdmap e470: 8 total, 8 up, 8 in 2024-02-05T07:44:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:59 smithi043 ceph-mon[41444]: pgmap v708: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-05T07:44:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:44:59 smithi043 ceph-mon[41444]: osdmap e470: 8 total, 8 up, 8 in 2024-02-05T07:44:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:59 smithi050 ceph-mon[33492]: pgmap v708: 292 pgs: 292 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 4.0 KiB/s wr, 9 op/s 2024-02-05T07:44:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:44:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:44:59 smithi050 ceph-mon[33492]: osdmap e470: 8 total, 8 up, 8 in 2024-02-05T07:45:01.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:45:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:45:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:01 smithi050 ceph-mon[33492]: osdmap e471: 8 total, 8 up, 8 in 2024-02-05T07:45:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:01 smithi050 ceph-mon[33492]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-05T07:45:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:01 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:01 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[29879]: osdmap e471: 8 total, 8 up, 8 in 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[29879]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[41444]: osdmap e471: 8 total, 8 up, 8 in 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[41444]: pgmap v711: 292 pgs: 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.5 KiB/s wr, 2 op/s 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:01 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:02 smithi050 ceph-mon[33492]: osdmap e472: 8 total, 8 up, 8 in 2024-02-05T07:45:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-85"}]: dispatch 2024-02-05T07:45:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-85"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[29879]: osdmap e472: 8 total, 8 up, 8 in 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-85"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-85"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[41444]: osdmap e472: 8 total, 8 up, 8 in 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-85"}]: dispatch 2024-02-05T07:45:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-85"}]: dispatch 2024-02-05T07:45:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:03 smithi043 ceph-mon[29879]: osdmap e473: 8 total, 8 up, 8 in 2024-02-05T07:45:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:03 smithi043 ceph-mon[29879]: pgmap v714: 260 pgs: 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:45:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:03 smithi043 ceph-mon[41444]: osdmap e473: 8 total, 8 up, 8 in 2024-02-05T07:45:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:03 smithi043 ceph-mon[41444]: pgmap v714: 260 pgs: 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:45:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:03 smithi050 ceph-mon[33492]: osdmap e473: 8 total, 8 up, 8 in 2024-02-05T07:45:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:03 smithi050 ceph-mon[33492]: pgmap v714: 260 pgs: 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:45:04.354 INFO:tasks.workunit.client.0.smithi043.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (17851 ms) 2024-02-05T07:45:04.354 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-05T07:45:04.354 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22682 ms) 2024-02-05T07:45:04.354 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-05T07:45:04.354 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3006 ms) 2024-02-05T07:45:04.354 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3016 ms) 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3014 ms) 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3008 ms) 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4011 ms) 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39944 ms) 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-05T07:45:04.355 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16998 ms) 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4025 ms) 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3025 ms) 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25064 ms) 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.356 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14897 ms) 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5021 ms) 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4039 ms) 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (25052 ms) 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10063 ms) 2024-02-05T07:45:04.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-05T07:45:04.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9051 ms) 2024-02-05T07:45:04.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-05T07:45:04.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:45:04.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9046 ms) 2024-02-05T07:45:04.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-05T07:45:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:04 smithi043 ceph-mon[29879]: osdmap e474: 8 total, 8 up, 8 in 2024-02-05T07:45:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:04 smithi043 ceph-mon[41444]: osdmap e474: 8 total, 8 up, 8 in 2024-02-05T07:45:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:04 smithi050 ceph-mon[33492]: osdmap e474: 8 total, 8 up, 8 in 2024-02-05T07:45:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[29879]: osdmap e475: 8 total, 8 up, 8 in 2024-02-05T07:45:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[29879]: pgmap v717: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[41444]: osdmap e475: 8 total, 8 up, 8 in 2024-02-05T07:45:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:05.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:05 smithi043 ceph-mon[41444]: pgmap v717: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:05 smithi050 ceph-mon[33492]: osdmap e475: 8 total, 8 up, 8 in 2024-02-05T07:45:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:05 smithi050 ceph-mon[33492]: pgmap v717: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:06.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:06 smithi043 ceph-mon[29879]: osdmap e476: 8 total, 8 up, 8 in 2024-02-05T07:45:06.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:06.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:06.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:06 smithi043 ceph-mon[41444]: osdmap e476: 8 total, 8 up, 8 in 2024-02-05T07:45:06.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:06.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:06 smithi050 ceph-mon[33492]: osdmap e476: 8 total, 8 up, 8 in 2024-02-05T07:45:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[29879]: osdmap e477: 8 total, 8 up, 8 in 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]: dispatch 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]: dispatch 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[29879]: pgmap v720: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[41444]: osdmap e477: 8 total, 8 up, 8 in 2024-02-05T07:45:07.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]: dispatch 2024-02-05T07:45:07.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]: dispatch 2024-02-05T07:45:07.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:07 smithi043 ceph-mon[41444]: pgmap v720: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T07:45:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:45:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:07 smithi050 ceph-mon[33492]: osdmap e477: 8 total, 8 up, 8 in 2024-02-05T07:45:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]: dispatch 2024-02-05T07:45:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]: dispatch 2024-02-05T07:45:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:07 smithi050 ceph-mon[33492]: pgmap v720: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]': finished 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[41444]: osdmap e478: 8 total, 8 up, 8 in 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]': finished 2024-02-05T07:45:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:45:08.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[29879]: osdmap e478: 8 total, 8 up, 8 in 2024-02-05T07:45:08.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:45:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:08 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_tier","val": "test-rados-api-smithi043-81993-89-test-flush"}]': finished 2024-02-05T07:45:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:45:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:08 smithi050 ceph-mon[33492]: osdmap e478: 8 total, 8 up, 8 in 2024-02-05T07:45:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:45:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:45:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[41444]: osdmap e479: 8 total, 8 up, 8 in 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[41444]: pgmap v723: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[41444]: osdmap e480: 8 total, 8 up, 8 in 2024-02-05T07:45:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[29879]: osdmap e479: 8 total, 8 up, 8 in 2024-02-05T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[29879]: pgmap v723: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:45:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:09 smithi043 ceph-mon[29879]: osdmap e480: 8 total, 8 up, 8 in 2024-02-05T07:45:09.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:45:09.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:09 smithi050 ceph-mon[33492]: osdmap e479: 8 total, 8 up, 8 in 2024-02-05T07:45:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:45:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:45:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:09 smithi050 ceph-mon[33492]: pgmap v723: 324 pgs: 32 creating+peering, 292 active+clean; 8.4 MiB data, 654 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-02-05T07:45:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:45:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:09 smithi050 ceph-mon[33492]: osdmap e480: 8 total, 8 up, 8 in 2024-02-05T07:45:11.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:45:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-mon[33492]: osdmap e481: 8 total, 8 up, 8 in 2024-02-05T07:45:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:11.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:11.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-87"}]: dispatch 2024-02-05T07:45:11.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-87"}]: dispatch 2024-02-05T07:45:11.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-mon[33492]: pgmap v726: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:45:11.574 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:11 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[29879]: osdmap e481: 8 total, 8 up, 8 in 2024-02-05T07:45:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-87"}]: dispatch 2024-02-05T07:45:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-87"}]: dispatch 2024-02-05T07:45:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[29879]: pgmap v726: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:45:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[41444]: osdmap e481: 8 total, 8 up, 8 in 2024-02-05T07:45:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-87"}]: dispatch 2024-02-05T07:45:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-87"}]: dispatch 2024-02-05T07:45:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[41444]: pgmap v726: 292 pgs: 292 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.2 KiB/s wr, 2 op/s 2024-02-05T07:45:11.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:11 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:12 smithi043 ceph-mon[29879]: osdmap e482: 8 total, 8 up, 8 in 2024-02-05T07:45:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:12 smithi043 ceph-mon[41444]: osdmap e482: 8 total, 8 up, 8 in 2024-02-05T07:45:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:12 smithi050 ceph-mon[33492]: osdmap e482: 8 total, 8 up, 8 in 2024-02-05T07:45:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[29879]: osdmap e483: 8 total, 8 up, 8 in 2024-02-05T07:45:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[29879]: pgmap v729: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:45:13.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[41444]: osdmap e483: 8 total, 8 up, 8 in 2024-02-05T07:45:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:13 smithi043 ceph-mon[41444]: pgmap v729: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:45:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:13 smithi050 ceph-mon[33492]: osdmap e483: 8 total, 8 up, 8 in 2024-02-05T07:45:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:13 smithi050 ceph-mon[33492]: pgmap v729: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:45:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[29879]: osdmap e484: 8 total, 8 up, 8 in 2024-02-05T07:45:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[41444]: osdmap e484: 8 total, 8 up, 8 in 2024-02-05T07:45:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:14.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:14 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:14 smithi050 ceph-mon[33492]: osdmap e484: 8 total, 8 up, 8 in 2024-02-05T07:45:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:45:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:45:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:15 smithi043 ceph-mon[29879]: osdmap e485: 8 total, 8 up, 8 in 2024-02-05T07:45:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:15 smithi043 ceph-mon[29879]: pgmap v732: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:45:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:15 smithi043 ceph-mon[41444]: osdmap e485: 8 total, 8 up, 8 in 2024-02-05T07:45:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:15 smithi043 ceph-mon[41444]: pgmap v732: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:45:15.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:15 smithi050 ceph-mon[33492]: osdmap e485: 8 total, 8 up, 8 in 2024-02-05T07:45:15.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:15 smithi050 ceph-mon[33492]: pgmap v732: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:16 smithi043 ceph-mon[29879]: osdmap e486: 8 total, 8 up, 8 in 2024-02-05T07:45:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:16 smithi043 ceph-mon[41444]: osdmap e486: 8 total, 8 up, 8 in 2024-02-05T07:45:16.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:16 smithi050 ceph-mon[33492]: osdmap e486: 8 total, 8 up, 8 in 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[41444]: osdmap e487: 8 total, 8 up, 8 in 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-90"}]: dispatch 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-90"}]: dispatch 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[41444]: pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[29879]: osdmap e487: 8 total, 8 up, 8 in 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-90"}]: dispatch 2024-02-05T07:45:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-90"}]: dispatch 2024-02-05T07:45:17.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:17 smithi043 ceph-mon[29879]: pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-02-05T07:45:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:17 smithi050 ceph-mon[33492]: osdmap e487: 8 total, 8 up, 8 in 2024-02-05T07:45:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-90"}]: dispatch 2024-02-05T07:45:17.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-90"}]: dispatch 2024-02-05T07:45:17.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:17 smithi050 ceph-mon[33492]: pgmap v735: 292 pgs: 292 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-02-05T07:45:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:18 smithi043 ceph-mon[29879]: osdmap e488: 8 total, 8 up, 8 in 2024-02-05T07:45:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:18 smithi043 ceph-mon[41444]: osdmap e488: 8 total, 8 up, 8 in 2024-02-05T07:45:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:18 smithi050 ceph-mon[33492]: osdmap e488: 8 total, 8 up, 8 in 2024-02-05T07:45:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:45:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:45:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[41444]: osdmap e489: 8 total, 8 up, 8 in 2024-02-05T07:45:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[41444]: pgmap v738: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:45:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[29879]: osdmap e489: 8 total, 8 up, 8 in 2024-02-05T07:45:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:19 smithi043 ceph-mon[29879]: pgmap v738: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:45:19.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:19 smithi050 ceph-mon[33492]: osdmap e489: 8 total, 8 up, 8 in 2024-02-05T07:45:19.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:19.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:19.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:19 smithi050 ceph-mon[33492]: pgmap v738: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[41444]: osdmap e490: 8 total, 8 up, 8 in 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[29879]: osdmap e490: 8 total, 8 up, 8 in 2024-02-05T07:45:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:20 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:20 smithi050 ceph-mon[33492]: osdmap e490: 8 total, 8 up, 8 in 2024-02-05T07:45:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:21.442 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:45:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:45:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:21 smithi043 ceph-mon[29879]: osdmap e491: 8 total, 8 up, 8 in 2024-02-05T07:45:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:21 smithi043 ceph-mon[29879]: pgmap v741: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:21 smithi043 ceph-mon[41444]: osdmap e491: 8 total, 8 up, 8 in 2024-02-05T07:45:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:21 smithi043 ceph-mon[41444]: pgmap v741: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:21 smithi050 ceph-mon[33492]: osdmap e491: 8 total, 8 up, 8 in 2024-02-05T07:45:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:21 smithi050 ceph-mon[33492]: pgmap v741: 292 pgs: 292 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:21.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[41444]: osdmap e492: 8 total, 8 up, 8 in 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-92"}]: dispatch 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-92"}]: dispatch 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[29879]: osdmap e492: 8 total, 8 up, 8 in 2024-02-05T07:45:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:22.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:22.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-92"}]: dispatch 2024-02-05T07:45:22.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-92"}]: dispatch 2024-02-05T07:45:22.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:22 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:22 smithi050 ceph-mon[33492]: osdmap e492: 8 total, 8 up, 8 in 2024-02-05T07:45:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-92"}]: dispatch 2024-02-05T07:45:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-92"}]: dispatch 2024-02-05T07:45:22.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:22 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:45:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: osdmap e493: 8 total, 8 up, 8 in 2024-02-05T07:45:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi043 to 3249M 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: Adjusting osd_memory_target on smithi050 to 3249M 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[29879]: pgmap v744: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: osdmap e493: 8 total, 8 up, 8 in 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi043 to 3249M 2024-02-05T07:45:23.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: Adjusting osd_memory_target on smithi050 to 3249M 2024-02-05T07:45:23.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:23.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:23.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.817 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:23 smithi043 ceph-mon[41444]: pgmap v744: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:45:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: osdmap e493: 8 total, 8 up, 8 in 2024-02-05T07:45:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi043 to 3249M 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: Adjusting osd_memory_target on smithi050 to 3249M 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:45:23.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:23 smithi050 ceph-mon[33492]: pgmap v744: 260 pgs: 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:45:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:24 smithi043 ceph-mon[41444]: osdmap e494: 8 total, 8 up, 8 in 2024-02-05T07:45:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:24 smithi043 ceph-mon[29879]: osdmap e494: 8 total, 8 up, 8 in 2024-02-05T07:45:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:24 smithi050 ceph-mon[33492]: osdmap e494: 8 total, 8 up, 8 in 2024-02-05T07:45:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[29879]: osdmap e495: 8 total, 8 up, 8 in 2024-02-05T07:45:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[29879]: pgmap v747: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[41444]: osdmap e495: 8 total, 8 up, 8 in 2024-02-05T07:45:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:25 smithi043 ceph-mon[41444]: pgmap v747: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:25 smithi050 ceph-mon[33492]: osdmap e495: 8 total, 8 up, 8 in 2024-02-05T07:45:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:25 smithi050 ceph-mon[33492]: pgmap v747: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 657 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:26 smithi043 ceph-mon[41444]: osdmap e496: 8 total, 8 up, 8 in 2024-02-05T07:45:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:26 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:26 smithi043 ceph-mon[29879]: osdmap e496: 8 total, 8 up, 8 in 2024-02-05T07:45:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:26 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:26 smithi050 ceph-mon[33492]: osdmap e496: 8 total, 8 up, 8 in 2024-02-05T07:45:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:26 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:45:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:27 smithi043 ceph-mon[29879]: osdmap e497: 8 total, 8 up, 8 in 2024-02-05T07:45:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:27 smithi043 ceph-mon[29879]: pgmap v750: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:45:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:27 smithi043 ceph-mon[41444]: osdmap e497: 8 total, 8 up, 8 in 2024-02-05T07:45:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:27 smithi043 ceph-mon[41444]: pgmap v750: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:45:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:27 smithi050 ceph-mon[33492]: osdmap e497: 8 total, 8 up, 8 in 2024-02-05T07:45:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:27 smithi050 ceph-mon[33492]: pgmap v750: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-05T07:45:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:29.498 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:45:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:45:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:45:29.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:29 smithi043 ceph-mon[29879]: pgmap v751: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:45:29.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:29 smithi043 ceph-mon[41444]: pgmap v751: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:45:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:29 smithi050 ceph-mon[33492]: pgmap v751: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-05T07:45:31.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:45:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:45:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:32 smithi043 ceph-mon[29879]: pgmap v752: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-05T07:45:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:32 smithi043 ceph-mon[41444]: pgmap v752: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-05T07:45:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:32 smithi050 ceph-mon[33492]: pgmap v752: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-02-05T07:45:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:34 smithi043 ceph-mon[29879]: pgmap v753: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T07:45:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:34 smithi043 ceph-mon[41444]: pgmap v753: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T07:45:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:34 smithi050 ceph-mon[33492]: pgmap v753: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-05T07:45:36.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:36 smithi043 ceph-mon[29879]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 985 B/s wr, 4 op/s 2024-02-05T07:45:36.365 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:36 smithi043 ceph-mon[41444]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 985 B/s wr, 4 op/s 2024-02-05T07:45:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:36 smithi050 ceph-mon[33492]: pgmap v754: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 985 B/s wr, 4 op/s 2024-02-05T07:45:37.322 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:45:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=cleanup t=2024-02-05T07:45:36.941987948Z level=info msg="Completed cleanup jobs" duration=2.35414ms 2024-02-05T07:45:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:38 smithi043 ceph-mon[29879]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 98 B/s wr, 2 op/s 2024-02-05T07:45:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:38 smithi043 ceph-mon[29879]: osdmap e498: 8 total, 8 up, 8 in 2024-02-05T07:45:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:38 smithi043 ceph-mon[41444]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 98 B/s wr, 2 op/s 2024-02-05T07:45:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:38 smithi043 ceph-mon[41444]: osdmap e498: 8 total, 8 up, 8 in 2024-02-05T07:45:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:38 smithi050 ceph-mon[33492]: pgmap v755: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 98 B/s wr, 2 op/s 2024-02-05T07:45:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:38 smithi050 ceph-mon[33492]: osdmap e498: 8 total, 8 up, 8 in 2024-02-05T07:45:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:45:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:45:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:45:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:40 smithi043 ceph-mon[29879]: pgmap v757: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-05T07:45:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:40 smithi043 ceph-mon[41444]: pgmap v757: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-05T07:45:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:40 smithi050 ceph-mon[33492]: pgmap v757: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 102 B/s wr, 2 op/s 2024-02-05T07:45:41.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:45:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:45:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:42 smithi043 ceph-mon[29879]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:42 smithi043 ceph-mon[41444]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:42 smithi050 ceph-mon[33492]: pgmap v758: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:44.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:44 smithi043 ceph-mon[41444]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:44 smithi043 ceph-mon[29879]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:44 smithi050 ceph-mon[33492]: pgmap v759: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:46.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:46 smithi043 ceph-mon[29879]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:46 smithi043 ceph-mon[29879]: osdmap e499: 8 total, 8 up, 8 in 2024-02-05T07:45:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:46 smithi043 ceph-mon[41444]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:46 smithi043 ceph-mon[41444]: osdmap e499: 8 total, 8 up, 8 in 2024-02-05T07:45:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:46 smithi050 ceph-mon[33492]: pgmap v760: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:46 smithi050 ceph-mon[33492]: osdmap e499: 8 total, 8 up, 8 in 2024-02-05T07:45:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[41444]: pgmap v762: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-94"}]: dispatch 2024-02-05T07:45:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-94"}]: dispatch 2024-02-05T07:45:48.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[29879]: pgmap v762: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:48.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:48.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:48.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-94"}]: dispatch 2024-02-05T07:45:48.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-94"}]: dispatch 2024-02-05T07:45:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:48 smithi050 ceph-mon[33492]: pgmap v762: 292 pgs: 292 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:45:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:45:48.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-94"}]: dispatch 2024-02-05T07:45:48.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-94"}]: dispatch 2024-02-05T07:45:49.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:49 smithi043 ceph-mon[29879]: osdmap e500: 8 total, 8 up, 8 in 2024-02-05T07:45:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:45:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:45:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:45:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:49 smithi043 ceph-mon[41444]: osdmap e500: 8 total, 8 up, 8 in 2024-02-05T07:45:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:49 smithi050 ceph-mon[33492]: osdmap e500: 8 total, 8 up, 8 in 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[29879]: pgmap v764: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[29879]: osdmap e501: 8 total, 8 up, 8 in 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[41444]: pgmap v764: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[41444]: osdmap e501: 8 total, 8 up, 8 in 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:50 smithi050 ceph-mon[33492]: pgmap v764: 260 pgs: 260 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:50 smithi050 ceph-mon[33492]: osdmap e501: 8 total, 8 up, 8 in 2024-02-05T07:45:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:45:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:51 smithi043 ceph-mon[29879]: osdmap e502: 8 total, 8 up, 8 in 2024-02-05T07:45:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:51 smithi043 ceph-mon[41444]: osdmap e502: 8 total, 8 up, 8 in 2024-02-05T07:45:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:51.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:45:51.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:51 smithi050 ceph-mon[33492]: osdmap e502: 8 total, 8 up, 8 in 2024-02-05T07:45:51.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:45:51.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:45:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:45:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:52 smithi043 ceph-mon[29879]: pgmap v767: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:52 smithi043 ceph-mon[29879]: osdmap e503: 8 total, 8 up, 8 in 2024-02-05T07:45:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:52 smithi043 ceph-mon[41444]: pgmap v767: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:52 smithi043 ceph-mon[41444]: osdmap e503: 8 total, 8 up, 8 in 2024-02-05T07:45:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:52 smithi050 ceph-mon[33492]: pgmap v767: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:45:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:52 smithi050 ceph-mon[33492]: osdmap e503: 8 total, 8 up, 8 in 2024-02-05T07:45:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:45:54.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:54 smithi043 ceph-mon[29879]: pgmap v769: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-05T07:45:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:54 smithi043 ceph-mon[41444]: pgmap v769: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-05T07:45:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:54 smithi050 ceph-mon[33492]: pgmap v769: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-02-05T07:45:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:56 smithi043 ceph-mon[41444]: pgmap v770: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 1 op/s 2024-02-05T07:45:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:56 smithi043 ceph-mon[29879]: pgmap v770: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 1 op/s 2024-02-05T07:45:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:56 smithi050 ceph-mon[33492]: pgmap v770: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 1 op/s 2024-02-05T07:45:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:58 smithi043 ceph-mon[41444]: pgmap v771: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 795 B/s wr, 2 op/s 2024-02-05T07:45:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:45:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:58 smithi043 ceph-mon[29879]: pgmap v771: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 795 B/s wr, 2 op/s 2024-02-05T07:45:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:45:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:58 smithi050 ceph-mon[33492]: pgmap v771: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 795 B/s wr, 2 op/s 2024-02-05T07:45:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:45:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:45:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:45:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:45:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:46:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:00 smithi043 ceph-mon[29879]: pgmap v772: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 704 B/s wr, 2 op/s 2024-02-05T07:46:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:00 smithi043 ceph-mon[41444]: pgmap v772: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 704 B/s wr, 2 op/s 2024-02-05T07:46:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:00 smithi050 ceph-mon[33492]: pgmap v772: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 704 B/s wr, 2 op/s 2024-02-05T07:46:01.563 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 07:46:01 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1[52148]: 2024-02-05T07:46:01.161+0000 7f7e04399700 -1 snap_mapper.add_oid found existing snaps mapped on 102:0336c865:test-rados-api-smithi043-81993-97::foo:21, removing 2024-02-05T07:46:01.572 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 07:46:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T07:46:01.161+0000 7f6af8705700 -1 snap_mapper.add_oid found existing snaps mapped on 102:0336c865:test-rados-api-smithi043-81993-97::foo:21, removing 2024-02-05T07:46:01.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:46:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:46:01.573 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 07:46:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T07:46:01.161+0000 7fb660afb700 -1 snap_mapper.add_oid found existing snaps mapped on 102:0336c865:test-rados-api-smithi043-81993-97::foo:21, removing 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[29879]: pgmap v773: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-96"}]: dispatch 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-96"}]: dispatch 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[41444]: pgmap v773: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-96"}]: dispatch 2024-02-05T07:46:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-96"}]: dispatch 2024-02-05T07:46:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:02 smithi050 ceph-mon[33492]: pgmap v773: 292 pgs: 292 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-02-05T07:46:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-96"}]: dispatch 2024-02-05T07:46:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-96"}]: dispatch 2024-02-05T07:46:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:03 smithi043 ceph-mon[29879]: osdmap e504: 8 total, 8 up, 8 in 2024-02-05T07:46:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:03 smithi043 ceph-mon[41444]: osdmap e504: 8 total, 8 up, 8 in 2024-02-05T07:46:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:03 smithi050 ceph-mon[33492]: osdmap e504: 8 total, 8 up, 8 in 2024-02-05T07:46:04.480 INFO:tasks.workunit.client.0.smithi043.stdout: a api_misc: started 0x55567c1c0640 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0790 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c0ed0 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1ce0 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2170 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c22c0 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c28b0 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1e10 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1f60 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2410 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2540 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2670 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c27a0 2024-02-05T07:46:04.481 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2e80 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c29e0 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2af0 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2c20 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2d50 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3450 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c2fb0 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3100 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3210 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3340 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3a20 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3580 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c36b0 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c37c0 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3910 2024-02-05T07:46:04.482 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4010 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3b70 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3cc0 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3dd0 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4180 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4610 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4760 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c3f00 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4c10 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4d20 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4e50 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4290 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c43a0 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c44f0 2024-02-05T07:46:04.483 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1720 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c1870 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4fa0 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c5410 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c5540 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c48b0 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c4a00 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c5b30 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55567c1c5690 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.484 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.485 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.486 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.487 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.487 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.487 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.487 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.487 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.487 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:46:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[41444]: pgmap v775: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:46:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[41444]: osdmap e505: 8 total, 8 up, 8 in 2024-02-05T07:46:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[29879]: pgmap v775: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:46:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[29879]: osdmap e505: 8 total, 8 up, 8 in 2024-02-05T07:46:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:04.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:04 smithi050 ceph-mon[33492]: pgmap v775: 260 pgs: 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:46:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:04 smithi050 ceph-mon[33492]: osdmap e505: 8 total, 8 up, 8 in 2024-02-05T07:46:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[29879]: osdmap e506: 8 total, 8 up, 8 in 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[41444]: osdmap e506: 8 total, 8 up, 8 in 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:05 smithi050 ceph-mon[33492]: osdmap e506: 8 total, 8 up, 8 in 2024-02-05T07:46:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[29879]: pgmap v778: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:46:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:46:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]: dispatch 2024-02-05T07:46:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[29879]: osdmap e507: 8 total, 8 up, 8 in 2024-02-05T07:46:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]: dispatch 2024-02-05T07:46:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[41444]: pgmap v778: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:46:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:46:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]: dispatch 2024-02-05T07:46:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[41444]: osdmap e507: 8 total, 8 up, 8 in 2024-02-05T07:46:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]: dispatch 2024-02-05T07:46:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:06 smithi050 ceph-mon[33492]: pgmap v778: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:46:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:46:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]: dispatch 2024-02-05T07:46:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:06 smithi050 ceph-mon[33492]: osdmap e507: 8 total, 8 up, 8 in 2024-02-05T07:46:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]: dispatch 2024-02-05T07:46:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:07.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]': finished 2024-02-05T07:46:07.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[29879]: osdmap e508: 8 total, 8 up, 8 in 2024-02-05T07:46:07.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:07.568 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:07.568 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:07.568 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]': finished 2024-02-05T07:46:07.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[41444]: osdmap e508: 8 total, 8 up, 8 in 2024-02-05T07:46:07.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:07.569 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:07 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-98", "mode": "writeback"}]': finished 2024-02-05T07:46:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:07 smithi050 ceph-mon[33492]: osdmap e508: 8 total, 8 up, 8 in 2024-02-05T07:46:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: pgmap v781: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:46:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]': finished 2024-02-05T07:46:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: osdmap e509: 8 total, 8 up, 8 in 2024-02-05T07:46:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:46:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:46:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: osdmap e510: 8 total, 8 up, 8 in 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: pgmap v781: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]': finished 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: osdmap e509: 8 total, 8 up, 8 in 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:46:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:08.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:46:08.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: osdmap e510: 8 total, 8 up, 8 in 2024-02-05T07:46:08.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T07:46:08.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T07:46:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: pgmap v781: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:46:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-98"}]': finished 2024-02-05T07:46:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: osdmap e509: 8 total, 8 up, 8 in 2024-02-05T07:46:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:46:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:46:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:46:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: osdmap e510: 8 total, 8 up, 8 in 2024-02-05T07:46:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T07:46:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]: dispatch 2024-02-05T07:46:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:46:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:46:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[29879]: pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]': finished 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[29879]: osdmap e511: 8 total, 8 up, 8 in 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[41444]: pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]': finished 2024-02-05T07:46:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[41444]: osdmap e511: 8 total, 8 up, 8 in 2024-02-05T07:46:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:10 smithi050 ceph-mon[33492]: pgmap v784: 292 pgs: 292 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:46:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_count","val": "1"}]': finished 2024-02-05T07:46:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:10 smithi050 ceph-mon[33492]: osdmap e511: 8 total, 8 up, 8 in 2024-02-05T07:46:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[29879]: osdmap e512: 8 total, 8 up, 8 in 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[41444]: osdmap e512: 8 total, 8 up, 8 in 2024-02-05T07:46:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T07:46:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T07:46:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:11 smithi050 ceph-mon[33492]: osdmap e512: 8 total, 8 up, 8 in 2024-02-05T07:46:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]: dispatch 2024-02-05T07:46:11.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:46:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[29879]: pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]': finished 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[29879]: osdmap e513: 8 total, 8 up, 8 in 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[41444]: pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]': finished 2024-02-05T07:46:12.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[41444]: osdmap e513: 8 total, 8 up, 8 in 2024-02-05T07:46:12.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:12.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:12.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:12 smithi050 ceph-mon[33492]: pgmap v787: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-98","var": "target_max_objects","val": "250"}]': finished 2024-02-05T07:46:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:12 smithi050 ceph-mon[33492]: osdmap e513: 8 total, 8 up, 8 in 2024-02-05T07:46:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:46:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[29879]: osdmap e514: 8 total, 8 up, 8 in 2024-02-05T07:46:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]': finished 2024-02-05T07:46:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[29879]: osdmap e515: 8 total, 8 up, 8 in 2024-02-05T07:46:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:46:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[41444]: osdmap e514: 8 total, 8 up, 8 in 2024-02-05T07:46:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]': finished 2024-02-05T07:46:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:13 smithi043 ceph-mon[41444]: osdmap e515: 8 total, 8 up, 8 in 2024-02-05T07:46:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:46:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:13 smithi050 ceph-mon[33492]: osdmap e514: 8 total, 8 up, 8 in 2024-02-05T07:46:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]: dispatch 2024-02-05T07:46:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-98"}]': finished 2024-02-05T07:46:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:13 smithi050 ceph-mon[33492]: osdmap e515: 8 total, 8 up, 8 in 2024-02-05T07:46:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:14 smithi043 ceph-mon[29879]: pgmap v790: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:14 smithi043 ceph-mon[41444]: pgmap v790: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:14 smithi050 ceph-mon[33492]: pgmap v790: 292 pgs: 292 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[29879]: osdmap e516: 8 total, 8 up, 8 in 2024-02-05T07:46:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[29879]: pgmap v793: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[29879]: osdmap e517: 8 total, 8 up, 8 in 2024-02-05T07:46:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[41444]: osdmap e516: 8 total, 8 up, 8 in 2024-02-05T07:46:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[41444]: pgmap v793: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:15.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[41444]: osdmap e517: 8 total, 8 up, 8 in 2024-02-05T07:46:15.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:15.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:15 smithi050 ceph-mon[33492]: osdmap e516: 8 total, 8 up, 8 in 2024-02-05T07:46:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:15 smithi050 ceph-mon[33492]: pgmap v793: 260 pgs: 260 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:15 smithi050 ceph-mon[33492]: osdmap e517: 8 total, 8 up, 8 in 2024-02-05T07:46:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[29879]: osdmap e518: 8 total, 8 up, 8 in 2024-02-05T07:46:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[41444]: osdmap e518: 8 total, 8 up, 8 in 2024-02-05T07:46:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:17 smithi050 ceph-mon[33492]: osdmap e518: 8 total, 8 up, 8 in 2024-02-05T07:46:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[29879]: pgmap v796: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[29879]: osdmap e519: 8 total, 8 up, 8 in 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[41444]: pgmap v796: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[41444]: osdmap e519: 8 total, 8 up, 8 in 2024-02-05T07:46:18.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:18 smithi050 ceph-mon[33492]: pgmap v796: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:46:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:18 smithi050 ceph-mon[33492]: osdmap e519: 8 total, 8 up, 8 in 2024-02-05T07:46:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:46:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:46:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]': finished 2024-02-05T07:46:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]: dispatch 2024-02-05T07:46:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[41444]: osdmap e520: 8 total, 8 up, 8 in 2024-02-05T07:46:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]: dispatch 2024-02-05T07:46:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[41444]: pgmap v799: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]': finished 2024-02-05T07:46:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]: dispatch 2024-02-05T07:46:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[29879]: osdmap e520: 8 total, 8 up, 8 in 2024-02-05T07:46:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]: dispatch 2024-02-05T07:46:19.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:19 smithi043 ceph-mon[29879]: pgmap v799: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-6", "overlaypool": "test-rados-api-smithi043-81993-100"}]': finished 2024-02-05T07:46:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]: dispatch 2024-02-05T07:46:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:19 smithi050 ceph-mon[33492]: osdmap e520: 8 total, 8 up, 8 in 2024-02-05T07:46:19.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]: dispatch 2024-02-05T07:46:19.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:19 smithi050 ceph-mon[33492]: pgmap v799: 292 pgs: 32 creating+peering, 260 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]': finished 2024-02-05T07:46:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[41444]: osdmap e521: 8 total, 8 up, 8 in 2024-02-05T07:46:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]': finished 2024-02-05T07:46:20.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[29879]: osdmap e521: 8 total, 8 up, 8 in 2024-02-05T07:46:20.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:20.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:20 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-100", "mode": "writeback"}]': finished 2024-02-05T07:46:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:20 smithi050 ceph-mon[33492]: osdmap e521: 8 total, 8 up, 8 in 2024-02-05T07:46:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:46:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[41444]: osdmap e522: 8 total, 8 up, 8 in 2024-02-05T07:46:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[41444]: pgmap v802: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:46:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[29879]: osdmap e522: 8 total, 8 up, 8 in 2024-02-05T07:46:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[29879]: pgmap v802: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:21.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:46:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:46:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-mon[33492]: osdmap e522: 8 total, 8 up, 8 in 2024-02-05T07:46:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-mon[33492]: pgmap v802: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[41444]: osdmap e523: 8 total, 8 up, 8 in 2024-02-05T07:46:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[29879]: osdmap e523: 8 total, 8 up, 8 in 2024-02-05T07:46:22.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:22.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:22.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:22 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:22 smithi050 ceph-mon[33492]: osdmap e523: 8 total, 8 up, 8 in 2024-02-05T07:46:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:22 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: osdmap e524: 8 total, 8 up, 8 in 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: pgmap v805: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-05T07:46:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[29879]: osdmap e525: 8 total, 8 up, 8 in 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: osdmap e524: 8 total, 8 up, 8 in 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: pgmap v805: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-05T07:46:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:23 smithi043 ceph-mon[41444]: osdmap e525: 8 total, 8 up, 8 in 2024-02-05T07:46:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:46:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: osdmap e524: 8 total, 8 up, 8 in 2024-02-05T07:46:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T07:46:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-05T07:46:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: pgmap v805: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:46:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-05T07:46:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:23 smithi050 ceph-mon[33492]: osdmap e525: 8 total, 8 up, 8 in 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:24 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:46:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:24 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:46:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:24 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:46:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:24 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:46:25.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[29879]: osdmap e526: 8 total, 8 up, 8 in 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[29879]: pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[41444]: osdmap e526: 8 total, 8 up, 8 in 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:25 smithi043 ceph-mon[41444]: pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]': finished 2024-02-05T07:46:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:25 smithi050 ceph-mon[33492]: osdmap e526: 8 total, 8 up, 8 in 2024-02-05T07:46:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]: dispatch 2024-02-05T07:46:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:25 smithi050 ceph-mon[33492]: pgmap v808: 292 pgs: 292 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]': finished 2024-02-05T07:46:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:26 smithi043 ceph-mon[29879]: osdmap e527: 8 total, 8 up, 8 in 2024-02-05T07:46:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]': finished 2024-02-05T07:46:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:26 smithi043 ceph-mon[41444]: osdmap e527: 8 total, 8 up, 8 in 2024-02-05T07:46:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-100"}]': finished 2024-02-05T07:46:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:26 smithi050 ceph-mon[33492]: osdmap e527: 8 total, 8 up, 8 in 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[29879]: osdmap e528: 8 total, 8 up, 8 in 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[29879]: pgmap v811: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[29879]: osdmap e529: 8 total, 8 up, 8 in 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[41444]: osdmap e528: 8 total, 8 up, 8 in 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[41444]: pgmap v811: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:27.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[41444]: osdmap e529: 8 total, 8 up, 8 in 2024-02-05T07:46:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:27 smithi050 ceph-mon[33492]: osdmap e528: 8 total, 8 up, 8 in 2024-02-05T07:46:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:27 smithi050 ceph-mon[33492]: pgmap v811: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:27 smithi050 ceph-mon[33492]: osdmap e529: 8 total, 8 up, 8 in 2024-02-05T07:46:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[29879]: osdmap e530: 8 total, 8 up, 8 in 2024-02-05T07:46:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:46:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:46:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:46:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:28.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:28.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[41444]: osdmap e530: 8 total, 8 up, 8 in 2024-02-05T07:46:28.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:46:28.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:46:28.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:46:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:28 smithi050 ceph-mon[33492]: osdmap e530: 8 total, 8 up, 8 in 2024-02-05T07:46:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-05T07:46:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:46:28.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-05T07:46:29.535 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:46:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[29879]: pgmap v814: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[29879]: osdmap e531: 8 total, 8 up, 8 in 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[41444]: pgmap v814: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:46:29.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[41444]: osdmap e531: 8 total, 8 up, 8 in 2024-02-05T07:46:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:46:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:46:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:29 smithi050 ceph-mon[33492]: pgmap v814: 292 pgs: 32 unknown, 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-05T07:46:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:29 smithi050 ceph-mon[33492]: osdmap e531: 8 total, 8 up, 8 in 2024-02-05T07:46:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:46:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-05T07:46:31.278 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[29879]: osdmap e532: 8 total, 8 up, 8 in 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[29879]: pgmap v817: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[41444]: osdmap e532: 8 total, 8 up, 8 in 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:46:31.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:46:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:31 smithi043 ceph-mon[41444]: pgmap v817: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:31.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:46:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:46:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-05T07:46:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:31 smithi050 ceph-mon[33492]: osdmap e532: 8 total, 8 up, 8 in 2024-02-05T07:46:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:46:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-05T07:46:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:31 smithi050 ceph-mon[33492]: pgmap v817: 292 pgs: 292 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[29879]: osdmap e533: 8 total, 8 up, 8 in 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-102"}]: dispatch 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-102"}]: dispatch 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:46:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[41444]: osdmap e533: 8 total, 8 up, 8 in 2024-02-05T07:46:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-102"}]: dispatch 2024-02-05T07:46:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-102"}]: dispatch 2024-02-05T07:46:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-05T07:46:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:32 smithi050 ceph-mon[33492]: osdmap e533: 8 total, 8 up, 8 in 2024-02-05T07:46:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-6"}]: dispatch 2024-02-05T07:46:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3836758274' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-102"}]: dispatch 2024-02-05T07:46:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-6", "tierpool": "test-rados-api-smithi043-81993-102"}]: dispatch 2024-02-05T07:46:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:33 smithi043 ceph-mon[29879]: osdmap e534: 8 total, 8 up, 8 in 2024-02-05T07:46:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:33 smithi043 ceph-mon[29879]: pgmap v820: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:33 smithi043 ceph-mon[41444]: osdmap e534: 8 total, 8 up, 8 in 2024-02-05T07:46:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:33 smithi043 ceph-mon[41444]: pgmap v820: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:33 smithi050 ceph-mon[33492]: osdmap e534: 8 total, 8 up, 8 in 2024-02-05T07:46:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:33 smithi050 ceph-mon[33492]: pgmap v820: 260 pgs: 260 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[29879]: osdmap e535: 8 total, 8 up, 8 in 2024-02-05T07:46:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[41444]: osdmap e535: 8 total, 8 up, 8 in 2024-02-05T07:46:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:34 smithi050 ceph-mon[33492]: osdmap e535: 8 total, 8 up, 8 in 2024-02-05T07:46:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:35.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:46:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[29879]: osdmap e536: 8 total, 8 up, 8 in 2024-02-05T07:46:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81993-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[29879]: pgmap v823: 228 pgs: 228 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:46:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[41444]: osdmap e536: 8 total, 8 up, 8 in 2024-02-05T07:46:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81993-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:35 smithi043 ceph-mon[41444]: pgmap v823: 228 pgs: 228 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:46:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:35 smithi050 ceph-mon[33492]: osdmap e536: 8 total, 8 up, 8 in 2024-02-05T07:46:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81993-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:46:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:35 smithi050 ceph-mon[33492]: pgmap v823: 228 pgs: 228 active+clean; 582 KiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:36 smithi043 ceph-mon[29879]: osdmap e537: 8 total, 8 up, 8 in 2024-02-05T07:46:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:36 smithi043 ceph-mon[41444]: osdmap e537: 8 total, 8 up, 8 in 2024-02-05T07:46:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:36 smithi050 ceph-mon[33492]: osdmap e537: 8 total, 8 up, 8 in 2024-02-05T07:46:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81993-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:46:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:37 smithi043 ceph-mon[29879]: osdmap e538: 8 total, 8 up, 8 in 2024-02-05T07:46:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:37 smithi043 ceph-mon[29879]: pgmap v826: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81993-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:46:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:37 smithi043 ceph-mon[41444]: osdmap e538: 8 total, 8 up, 8 in 2024-02-05T07:46:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:37 smithi043 ceph-mon[41444]: pgmap v826: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81993-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:46:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:37 smithi050 ceph-mon[33492]: osdmap e538: 8 total, 8 up, 8 in 2024-02-05T07:46:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:37 smithi050 ceph-mon[33492]: pgmap v826: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:38 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:38 smithi050 ceph-mon[33492]: osdmap e539: 8 total, 8 up, 8 in 2024-02-05T07:46:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:38 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:38 smithi043 ceph-mon[29879]: osdmap e539: 8 total, 8 up, 8 in 2024-02-05T07:46:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:38 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:38 smithi043 ceph-mon[41444]: osdmap e539: 8 total, 8 up, 8 in 2024-02-05T07:46:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:46:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[29879]: osdmap e540: 8 total, 8 up, 8 in 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[29879]: pgmap v829: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[29879]: osdmap e541: 8 total, 8 up, 8 in 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[41444]: osdmap e540: 8 total, 8 up, 8 in 2024-02-05T07:46:40.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[41444]: pgmap v829: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:40.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:39 smithi043 ceph-mon[41444]: osdmap e541: 8 total, 8 up, 8 in 2024-02-05T07:46:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:39 smithi050 ceph-mon[33492]: osdmap e540: 8 total, 8 up, 8 in 2024-02-05T07:46:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:39 smithi050 ceph-mon[33492]: pgmap v829: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:40.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:39 smithi050 ceph-mon[33492]: osdmap e541: 8 total, 8 up, 8 in 2024-02-05T07:46:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:41 smithi050 ceph-mon[33492]: osdmap e542: 8 total, 8 up, 8 in 2024-02-05T07:46:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:41 smithi050 ceph-mon[33492]: pgmap v832: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:41.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:46:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[41444]: osdmap e542: 8 total, 8 up, 8 in 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[41444]: pgmap v832: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[29879]: osdmap e542: 8 total, 8 up, 8 in 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:41 smithi043 ceph-mon[29879]: pgmap v832: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[29879]: osdmap e543: 8 total, 8 up, 8 in 2024-02-05T07:46:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:42.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[41444]: osdmap e543: 8 total, 8 up, 8 in 2024-02-05T07:46:42.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:42.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:42 smithi050 ceph-mon[33492]: osdmap e543: 8 total, 8 up, 8 in 2024-02-05T07:46:42.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:42.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: osdmap e544: 8 total, 8 up, 8 in 2024-02-05T07:46:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: pgmap v835: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[29879]: osdmap e545: 8 total, 8 up, 8 in 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: osdmap e544: 8 total, 8 up, 8 in 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: pgmap v835: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:43 smithi043 ceph-mon[41444]: osdmap e545: 8 total, 8 up, 8 in 2024-02-05T07:46:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: osdmap e544: 8 total, 8 up, 8 in 2024-02-05T07:46:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: pgmap v835: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:46:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-107", "overlaypool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:43 smithi050 ceph-mon[33492]: osdmap e545: 8 total, 8 up, 8 in 2024-02-05T07:46:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]': finished 2024-02-05T07:46:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[29879]: osdmap e546: 8 total, 8 up, 8 in 2024-02-05T07:46:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]': finished 2024-02-05T07:46:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:44 smithi043 ceph-mon[41444]: osdmap e546: 8 total, 8 up, 8 in 2024-02-05T07:46:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:44 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:44 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:44 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-107-cache", "mode": "writeback"}]': finished 2024-02-05T07:46:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:44 smithi050 ceph-mon[33492]: osdmap e546: 8 total, 8 up, 8 in 2024-02-05T07:46:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]: dispatch 2024-02-05T07:46:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:45 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]: dispatch 2024-02-05T07:46:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:45 smithi043 ceph-mon[29879]: pgmap v838: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]: dispatch 2024-02-05T07:46:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:45 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]: dispatch 2024-02-05T07:46:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:45 smithi043 ceph-mon[41444]: pgmap v838: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]: dispatch 2024-02-05T07:46:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:45 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]: dispatch 2024-02-05T07:46:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:45 smithi050 ceph-mon[33492]: pgmap v838: 300 pgs: 32 creating+peering, 268 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]': finished 2024-02-05T07:46:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[29879]: osdmap e547: 8 total, 8 up, 8 in 2024-02-05T07:46:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]': finished 2024-02-05T07:46:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[41444]: osdmap e547: 8 total, 8 up, 8 in 2024-02-05T07:46:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:46 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-107"}]': finished 2024-02-05T07:46:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:46 smithi050 ceph-mon[33492]: osdmap e547: 8 total, 8 up, 8 in 2024-02-05T07:46:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2988312807' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:46 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]: dispatch 2024-02-05T07:46:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[29879]: osdmap e548: 8 total, 8 up, 8 in 2024-02-05T07:46:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[29879]: pgmap v841: 300 pgs: 300 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:46:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[41444]: osdmap e548: 8 total, 8 up, 8 in 2024-02-05T07:46:47.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:47 smithi043 ceph-mon[41444]: pgmap v841: 300 pgs: 300 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:46:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:47 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:46:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-107", "tierpool": "test-rados-api-smithi043-81993-107-cache"}]': finished 2024-02-05T07:46:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:47 smithi050 ceph-mon[33492]: osdmap e548: 8 total, 8 up, 8 in 2024-02-05T07:46:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:47 smithi050 ceph-mon[33492]: pgmap v841: 300 pgs: 300 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:46:48.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:48 smithi043 ceph-mon[29879]: osdmap e549: 8 total, 8 up, 8 in 2024-02-05T07:46:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:48 smithi043 ceph-mon[41444]: osdmap e549: 8 total, 8 up, 8 in 2024-02-05T07:46:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:48 smithi050 ceph-mon[33492]: osdmap e549: 8 total, 8 up, 8 in 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:46:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: osdmap e550: 8 total, 8 up, 8 in 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[41444]: pgmap v844: 236 pgs: 236 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: osdmap e550: 8 total, 8 up, 8 in 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:49.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:49 smithi043 ceph-mon[29879]: pgmap v844: 236 pgs: 236 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:49.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: osdmap e550: 8 total, 8 up, 8 in 2024-02-05T07:46:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:46:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:49 smithi050 ceph-mon[33492]: pgmap v844: 236 pgs: 236 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:46:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[29879]: osdmap e551: 8 total, 8 up, 8 in 2024-02-05T07:46:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:51.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:51.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:46:51.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[41444]: osdmap e551: 8 total, 8 up, 8 in 2024-02-05T07:46:51.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:51.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:51.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:46:51.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:50 smithi050 ceph-mon[33492]: osdmap e551: 8 total, 8 up, 8 in 2024-02-05T07:46:51.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:51.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:46:51.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:46:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:46:52.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[29879]: osdmap e552: 8 total, 8 up, 8 in 2024-02-05T07:46:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[29879]: pgmap v847: 236 pgs: 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:46:52.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[29879]: osdmap e553: 8 total, 8 up, 8 in 2024-02-05T07:46:52.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[41444]: osdmap e552: 8 total, 8 up, 8 in 2024-02-05T07:46:52.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[41444]: pgmap v847: 236 pgs: 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:52.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:46:52.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:51 smithi043 ceph-mon[41444]: osdmap e553: 8 total, 8 up, 8 in 2024-02-05T07:46:52.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:51 smithi050 ceph-mon[33492]: osdmap e552: 8 total, 8 up, 8 in 2024-02-05T07:46:52.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:51 smithi050 ceph-mon[33492]: pgmap v847: 236 pgs: 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:52.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:46:52.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:51 smithi050 ceph-mon[33492]: osdmap e553: 8 total, 8 up, 8 in 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[41444]: osdmap e554: 8 total, 8 up, 8 in 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[29879]: osdmap e554: 8 total, 8 up, 8 in 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:53.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:53.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:46:53.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:52 smithi050 ceph-mon[33492]: osdmap e554: 8 total, 8 up, 8 in 2024-02-05T07:46:53.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:53.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[29879]: pgmap v850: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[29879]: osdmap e555: 8 total, 8 up, 8 in 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[41444]: pgmap v850: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:54.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[41444]: osdmap e555: 8 total, 8 up, 8 in 2024-02-05T07:46:54.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:54.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:54.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:53 smithi050 ceph-mon[33492]: pgmap v850: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:54.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:46:54.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:53 smithi050 ceph-mon[33492]: osdmap e555: 8 total, 8 up, 8 in 2024-02-05T07:46:54.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:54.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[29879]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[29879]: osdmap e556: 8 total, 8 up, 8 in 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[41444]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[41444]: osdmap e556: 8 total, 8 up, 8 in 2024-02-05T07:46:55.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:55.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:54 smithi050 ceph-mon[33492]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:46:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:46:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:54 smithi050 ceph-mon[33492]: osdmap e556: 8 total, 8 up, 8 in 2024-02-05T07:46:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:55.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:46:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[29879]: pgmap v853: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:46:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[29879]: osdmap e557: 8 total, 8 up, 8 in 2024-02-05T07:46:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:56.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[41444]: pgmap v853: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:56.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:46:56.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[41444]: osdmap e557: 8 total, 8 up, 8 in 2024-02-05T07:46:56.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:56.064 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:55 smithi050 ceph-mon[33492]: pgmap v853: 276 pgs: 40 unknown, 236 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:46:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-109", "overlaypool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:46:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:55 smithi050 ceph-mon[33492]: osdmap e557: 8 total, 8 up, 8 in 2024-02-05T07:46:56.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:56.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]: dispatch 2024-02-05T07:46:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]': finished 2024-02-05T07:46:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[29879]: osdmap e558: 8 total, 8 up, 8 in 2024-02-05T07:46:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]': finished 2024-02-05T07:46:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[41444]: osdmap e558: 8 total, 8 up, 8 in 2024-02-05T07:46:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:57 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:46:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-109-cache", "mode": "writeback"}]': finished 2024-02-05T07:46:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:57 smithi050 ceph-mon[33492]: osdmap e558: 8 total, 8 up, 8 in 2024-02-05T07:46:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:57.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:46:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[29879]: pgmap v856: 276 pgs: 276 active+clean; 582 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:46:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[29879]: osdmap e559: 8 total, 8 up, 8 in 2024-02-05T07:46:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:46:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[41444]: pgmap v856: 276 pgs: 276 active+clean; 582 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:46:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[41444]: osdmap e559: 8 total, 8 up, 8 in 2024-02-05T07:46:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:46:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:58 smithi050 ceph-mon[33492]: pgmap v856: 276 pgs: 276 active+clean; 582 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:46:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:58 smithi050 ceph-mon[33492]: osdmap e559: 8 total, 8 up, 8 in 2024-02-05T07:46:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:46:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:46:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:46:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:46:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:46:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[29879]: osdmap e560: 8 total, 8 up, 8 in 2024-02-05T07:46:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[29879]: pgmap v859: 276 pgs: 276 active+clean; 582 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[41444]: osdmap e560: 8 total, 8 up, 8 in 2024-02-05T07:46:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:46:59 smithi043 ceph-mon[41444]: pgmap v859: 276 pgs: 276 active+clean; 582 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:46:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:46:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:59 smithi050 ceph-mon[33492]: osdmap e560: 8 total, 8 up, 8 in 2024-02-05T07:46:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:59.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:46:59.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:46:59 smithi050 ceph-mon[33492]: pgmap v859: 276 pgs: 276 active+clean; 582 KiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[29879]: osdmap e561: 8 total, 8 up, 8 in 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[41444]: osdmap e561: 8 total, 8 up, 8 in 2024-02-05T07:47:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T07:47:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T07:47:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:00 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:47:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:00 smithi050 ceph-mon[33492]: osdmap e561: 8 total, 8 up, 8 in 2024-02-05T07:47:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T07:47:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-05T07:47:01.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:47:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:47:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-05T07:47:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:01 smithi050 ceph-mon[33492]: osdmap e562: 8 total, 8 up, 8 in 2024-02-05T07:47:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:01 smithi050 ceph-mon[33492]: pgmap v862: 276 pgs: 276 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:01 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-05T07:47:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[29879]: osdmap e562: 8 total, 8 up, 8 in 2024-02-05T07:47:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[29879]: pgmap v862: 276 pgs: 276 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-05T07:47:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[41444]: osdmap e562: 8 total, 8 up, 8 in 2024-02-05T07:47:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[41444]: pgmap v862: 276 pgs: 276 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:01 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[29879]: osdmap e563: 8 total, 8 up, 8 in 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[41444]: osdmap e563: 8 total, 8 up, 8 in 2024-02-05T07:47:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:47:02.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:47:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:02 smithi050 ceph-mon[33492]: osdmap e563: 8 total, 8 up, 8 in 2024-02-05T07:47:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:47:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]: dispatch 2024-02-05T07:47:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:47:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:03 smithi043 ceph-mon[29879]: osdmap e564: 8 total, 8 up, 8 in 2024-02-05T07:47:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:03 smithi043 ceph-mon[29879]: pgmap v865: 276 pgs: 276 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:47:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:03 smithi043 ceph-mon[41444]: osdmap e564: 8 total, 8 up, 8 in 2024-02-05T07:47:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:03 smithi043 ceph-mon[41444]: pgmap v865: 276 pgs: 276 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-109", "tierpool": "test-rados-api-smithi043-81993-109-cache"}]': finished 2024-02-05T07:47:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:03 smithi050 ceph-mon[33492]: osdmap e564: 8 total, 8 up, 8 in 2024-02-05T07:47:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:03 smithi050 ceph-mon[33492]: pgmap v865: 276 pgs: 276 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:04 smithi043 ceph-mon[41444]: osdmap e565: 8 total, 8 up, 8 in 2024-02-05T07:47:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:04 smithi043 ceph-mon[29879]: osdmap e565: 8 total, 8 up, 8 in 2024-02-05T07:47:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:04 smithi050 ceph-mon[33492]: osdmap e565: 8 total, 8 up, 8 in 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[29879]: osdmap e566: 8 total, 8 up, 8 in 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[29879]: pgmap v868: 236 pgs: 236 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[41444]: osdmap e566: 8 total, 8 up, 8 in 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:05 smithi043 ceph-mon[41444]: pgmap v868: 236 pgs: 236 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:05 smithi050 ceph-mon[33492]: osdmap e566: 8 total, 8 up, 8 in 2024-02-05T07:47:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:05 smithi050 ceph-mon[33492]: pgmap v868: 236 pgs: 236 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:06.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:06.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[41444]: osdmap e567: 8 total, 8 up, 8 in 2024-02-05T07:47:06.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:06.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:06.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:06.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:06.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[29879]: osdmap e567: 8 total, 8 up, 8 in 2024-02-05T07:47:06.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:06.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:06.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:06 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:06 smithi050 ceph-mon[33492]: osdmap e567: 8 total, 8 up, 8 in 2024-02-05T07:47:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1127299708' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:06.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]: dispatch 2024-02-05T07:47:06.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:06 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:07 smithi043 ceph-mon[29879]: osdmap e568: 8 total, 8 up, 8 in 2024-02-05T07:47:07.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:07 smithi043 ceph-mon[29879]: pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:07.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:07.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:07 smithi043 ceph-mon[41444]: osdmap e568: 8 total, 8 up, 8 in 2024-02-05T07:47:07.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:07 smithi043 ceph-mon[41444]: pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-109"}]': finished 2024-02-05T07:47:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:07 smithi050 ceph-mon[33492]: osdmap e568: 8 total, 8 up, 8 in 2024-02-05T07:47:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:07 smithi050 ceph-mon[33492]: pgmap v871: 236 pgs: 236 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[29879]: osdmap e569: 8 total, 8 up, 8 in 2024-02-05T07:47:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:47:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:47:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[29879]: osdmap e570: 8 total, 8 up, 8 in 2024-02-05T07:47:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:47:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[41444]: osdmap e569: 8 total, 8 up, 8 in 2024-02-05T07:47:08.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:47:08.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:47:08.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[41444]: osdmap e570: 8 total, 8 up, 8 in 2024-02-05T07:47:08.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:47:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:08 smithi050 ceph-mon[33492]: osdmap e569: 8 total, 8 up, 8 in 2024-02-05T07:47:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:47:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:47:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:08 smithi050 ceph-mon[33492]: osdmap e570: 8 total, 8 up, 8 in 2024-02-05T07:47:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]: dispatch 2024-02-05T07:47:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:09 smithi043 ceph-mon[29879]: pgmap v874: 228 pgs: 228 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:47:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:09 smithi043 ceph-mon[29879]: osdmap e571: 8 total, 8 up, 8 in 2024-02-05T07:47:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:47:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:47:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:47:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:09 smithi043 ceph-mon[41444]: pgmap v874: 228 pgs: 228 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:47:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:09 smithi043 ceph-mon[41444]: osdmap e571: 8 total, 8 up, 8 in 2024-02-05T07:47:09.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:09 smithi050 ceph-mon[33492]: pgmap v874: 228 pgs: 228 active+clean; 582 KiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:09.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3281828601' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81993-104"}]': finished 2024-02-05T07:47:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:09 smithi050 ceph-mon[33492]: osdmap e571: 8 total, 8 up, 8 in 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:47:10.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:47:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:10.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:47:10.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-05T07:47:11.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:47:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[41444]: osdmap e572: 8 total, 8 up, 8 in 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[41444]: pgmap v877: 228 pgs: 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:47:12.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[29879]: osdmap e572: 8 total, 8 up, 8 in 2024-02-05T07:47:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[29879]: pgmap v877: 228 pgs: 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:12.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:11 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:12.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81993-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-05T07:47:12.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:11 smithi050 ceph-mon[33492]: osdmap e572: 8 total, 8 up, 8 in 2024-02-05T07:47:12.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:11 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:11 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:11 smithi050 ceph-mon[33492]: pgmap v877: 228 pgs: 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-05T07:47:12.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:11 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[41444]: osdmap e573: 8 total, 8 up, 8 in 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[41444]: osdmap e574: 8 total, 8 up, 8 in 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[29879]: osdmap e573: 8 total, 8 up, 8 in 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:13.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:12 smithi043 ceph-mon[29879]: osdmap e574: 8 total, 8 up, 8 in 2024-02-05T07:47:13.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:13.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:12 smithi050 ceph-mon[33492]: osdmap e573: 8 total, 8 up, 8 in 2024-02-05T07:47:13.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81993-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:13.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:12 smithi050 ceph-mon[33492]: osdmap e574: 8 total, 8 up, 8 in 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[41444]: pgmap v880: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[41444]: osdmap e575: 8 total, 8 up, 8 in 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[29879]: pgmap v880: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:14.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:14.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:14.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[29879]: osdmap e575: 8 total, 8 up, 8 in 2024-02-05T07:47:14.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:14.314 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:13 smithi050 ceph-mon[33492]: pgmap v880: 236 pgs: 8 unknown, 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:14.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:13 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:14.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:14.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:13 smithi050 ceph-mon[33492]: osdmap e575: 8 total, 8 up, 8 in 2024-02-05T07:47:14.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:14.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:15.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:15.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[29879]: osdmap e576: 8 total, 8 up, 8 in 2024-02-05T07:47:15.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:15.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:15.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[29879]: pgmap v883: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:15.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:15.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[41444]: osdmap e576: 8 total, 8 up, 8 in 2024-02-05T07:47:15.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:15.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:15.816 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:15 smithi043 ceph-mon[41444]: pgmap v883: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:15 smithi050 ceph-mon[33492]: osdmap e576: 8 total, 8 up, 8 in 2024-02-05T07:47:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:15 smithi050 ceph-mon[33492]: pgmap v883: 268 pgs: 40 unknown, 228 active+clean; 582 KiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[29879]: osdmap e577: 8 total, 8 up, 8 in 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[41444]: osdmap e577: 8 total, 8 up, 8 in 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:16 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:16 smithi050 ceph-mon[33492]: osdmap e577: 8 total, 8 up, 8 in 2024-02-05T07:47:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:16 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]': finished 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[41444]: osdmap e578: 8 total, 8 up, 8 in 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[41444]: pgmap v886: 268 pgs: 268 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]': finished 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[29879]: osdmap e578: 8 total, 8 up, 8 in 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:17 smithi043 ceph-mon[29879]: pgmap v886: 268 pgs: 268 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-112"}]': finished 2024-02-05T07:47:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:17 smithi050 ceph-mon[33492]: osdmap e578: 8 total, 8 up, 8 in 2024-02-05T07:47:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:17.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:17 smithi050 ceph-mon[33492]: pgmap v886: 268 pgs: 268 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[29879]: osdmap e579: 8 total, 8 up, 8 in 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[41444]: osdmap e579: 8 total, 8 up, 8 in 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:18 smithi050 ceph-mon[33492]: osdmap e579: 8 total, 8 up, 8 in 2024-02-05T07:47:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]: dispatch 2024-02-05T07:47:19.498 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:47:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:47:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:47:19.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]': finished 2024-02-05T07:47:19.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:19 smithi043 ceph-mon[41444]: osdmap e580: 8 total, 8 up, 8 in 2024-02-05T07:47:19.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:19 smithi043 ceph-mon[41444]: pgmap v889: 268 pgs: 268 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]': finished 2024-02-05T07:47:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:19 smithi043 ceph-mon[29879]: osdmap e580: 8 total, 8 up, 8 in 2024-02-05T07:47:19.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:19 smithi043 ceph-mon[29879]: pgmap v889: 268 pgs: 268 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:19.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-112"}]': finished 2024-02-05T07:47:19.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:19 smithi050 ceph-mon[33492]: osdmap e580: 8 total, 8 up, 8 in 2024-02-05T07:47:19.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:19 smithi050 ceph-mon[33492]: pgmap v889: 268 pgs: 268 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:20.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:20 smithi043 ceph-mon[29879]: osdmap e581: 8 total, 8 up, 8 in 2024-02-05T07:47:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:20 smithi043 ceph-mon[41444]: osdmap e581: 8 total, 8 up, 8 in 2024-02-05T07:47:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:20 smithi050 ceph-mon[33492]: osdmap e581: 8 total, 8 up, 8 in 2024-02-05T07:47:21.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:47:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:47:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:21 smithi050 ceph-mon[33492]: osdmap e582: 8 total, 8 up, 8 in 2024-02-05T07:47:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:21 smithi050 ceph-mon[33492]: pgmap v892: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:21 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[29879]: osdmap e582: 8 total, 8 up, 8 in 2024-02-05T07:47:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[29879]: pgmap v892: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:21.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[41444]: osdmap e582: 8 total, 8 up, 8 in 2024-02-05T07:47:21.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[41444]: pgmap v892: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:21.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:21 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[29879]: osdmap e583: 8 total, 8 up, 8 in 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[41444]: osdmap e583: 8 total, 8 up, 8 in 2024-02-05T07:47:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:22.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:22 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:22 smithi050 ceph-mon[33492]: osdmap e583: 8 total, 8 up, 8 in 2024-02-05T07:47:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[29879]: osdmap e584: 8 total, 8 up, 8 in 2024-02-05T07:47:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[29879]: pgmap v895: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[41444]: osdmap e584: 8 total, 8 up, 8 in 2024-02-05T07:47:23.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:23.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:23.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:23 smithi043 ceph-mon[41444]: pgmap v895: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:23 smithi050 ceph-mon[33492]: osdmap e584: 8 total, 8 up, 8 in 2024-02-05T07:47:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:23 smithi050 ceph-mon[33492]: pgmap v895: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]': finished 2024-02-05T07:47:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[29879]: osdmap e585: 8 total, 8 up, 8 in 2024-02-05T07:47:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]: dispatch 2024-02-05T07:47:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]: dispatch 2024-02-05T07:47:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:24.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]': finished 2024-02-05T07:47:24.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[41444]: osdmap e585: 8 total, 8 up, 8 in 2024-02-05T07:47:24.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]: dispatch 2024-02-05T07:47:24.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]: dispatch 2024-02-05T07:47:24.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:24 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-114"}]': finished 2024-02-05T07:47:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:24 smithi050 ceph-mon[33492]: osdmap e585: 8 total, 8 up, 8 in 2024-02-05T07:47:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]: dispatch 2024-02-05T07:47:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]: dispatch 2024-02-05T07:47:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:24 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:47:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]': finished 2024-02-05T07:47:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[29879]: osdmap e586: 8 total, 8 up, 8 in 2024-02-05T07:47:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[29879]: pgmap v898: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]': finished 2024-02-05T07:47:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[41444]: osdmap e586: 8 total, 8 up, 8 in 2024-02-05T07:47:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:25.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:25 smithi043 ceph-mon[41444]: pgmap v898: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:25 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:25.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-114", "mode": "writeback"}]': finished 2024-02-05T07:47:25.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:25 smithi050 ceph-mon[33492]: osdmap e586: 8 total, 8 up, 8 in 2024-02-05T07:47:25.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:25.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:25.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:25 smithi050 ceph-mon[33492]: pgmap v898: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[29879]: osdmap e587: 8 total, 8 up, 8 in 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[41444]: osdmap e587: 8 total, 8 up, 8 in 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:26 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:26 smithi050 ceph-mon[33492]: osdmap e587: 8 total, 8 up, 8 in 2024-02-05T07:47:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]: dispatch 2024-02-05T07:47:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:26 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]': finished 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[29879]: osdmap e588: 8 total, 8 up, 8 in 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[29879]: pgmap v901: 268 pgs: 268 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]': finished 2024-02-05T07:47:27.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[41444]: osdmap e588: 8 total, 8 up, 8 in 2024-02-05T07:47:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:27 smithi043 ceph-mon[41444]: pgmap v901: 268 pgs: 268 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:27 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-114"}]': finished 2024-02-05T07:47:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:27 smithi050 ceph-mon[33492]: osdmap e588: 8 total, 8 up, 8 in 2024-02-05T07:47:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:27 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:27 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:27 smithi050 ceph-mon[33492]: pgmap v901: 268 pgs: 268 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:47:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:28 smithi043 ceph-mon[29879]: osdmap e589: 8 total, 8 up, 8 in 2024-02-05T07:47:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:28 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:28 smithi043 ceph-mon[41444]: osdmap e589: 8 total, 8 up, 8 in 2024-02-05T07:47:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:28 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:28 smithi050 ceph-mon[33492]: osdmap e589: 8 total, 8 up, 8 in 2024-02-05T07:47:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:28 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:47:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: osdmap e590: 8 total, 8 up, 8 in 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: pgmap v904: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:29.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: osdmap e590: 8 total, 8 up, 8 in 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: pgmap v904: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:29.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:29 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: osdmap e590: 8 total, 8 up, 8 in 2024-02-05T07:47:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: pgmap v904: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:47:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:47:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:47:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:29 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:30.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:30 smithi050 ceph-mon[33492]: osdmap e591: 8 total, 8 up, 8 in 2024-02-05T07:47:31.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:31.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:30 smithi043 ceph-mon[29879]: osdmap e591: 8 total, 8 up, 8 in 2024-02-05T07:47:31.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:31.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:30 smithi043 ceph-mon[41444]: osdmap e591: 8 total, 8 up, 8 in 2024-02-05T07:47:31.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:47:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[41444]: osdmap e592: 8 total, 8 up, 8 in 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[41444]: pgmap v907: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[29879]: osdmap e592: 8 total, 8 up, 8 in 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:32.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[29879]: pgmap v907: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:47:32.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:31 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:32.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:31 smithi050 ceph-mon[33492]: osdmap e592: 8 total, 8 up, 8 in 2024-02-05T07:47:32.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:32.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:32.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:31 smithi050 ceph-mon[33492]: pgmap v907: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-05T07:47:32.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:31 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: osdmap e593: 8 total, 8 up, 8 in 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]': finished 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: osdmap e594: 8 total, 8 up, 8 in 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]: dispatch 2024-02-05T07:47:33.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: osdmap e593: 8 total, 8 up, 8 in 2024-02-05T07:47:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]': finished 2024-02-05T07:47:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: osdmap e594: 8 total, 8 up, 8 in 2024-02-05T07:47:33.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]: dispatch 2024-02-05T07:47:33.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:33.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:33.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: osdmap e593: 8 total, 8 up, 8 in 2024-02-05T07:47:33.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:33.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-116"}]': finished 2024-02-05T07:47:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: osdmap e594: 8 total, 8 up, 8 in 2024-02-05T07:47:33.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]: dispatch 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]: dispatch 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[41444]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]': finished 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[41444]: osdmap e595: 8 total, 8 up, 8 in 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]: dispatch 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[29879]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:34.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]': finished 2024-02-05T07:47:34.064 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:33 smithi043 ceph-mon[29879]: osdmap e595: 8 total, 8 up, 8 in 2024-02-05T07:47:34.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]: dispatch 2024-02-05T07:47:34.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:33 smithi050 ceph-mon[33492]: pgmap v910: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-05T07:47:34.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:33 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:34.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-116", "mode": "writeback"}]': finished 2024-02-05T07:47:34.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:33 smithi050 ceph-mon[33492]: osdmap e595: 8 total, 8 up, 8 in 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.1"}]: dispatch 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[29879]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.1"}]: dispatch 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[29879]: 325.1 scrub starts 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[29879]: 325.1 scrub ok 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.1"}]: dispatch 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[41444]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.1"}]: dispatch 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[41444]: 325.1 scrub starts 2024-02-05T07:47:35.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:34 smithi043 ceph-mon[41444]: 325.1 scrub ok 2024-02-05T07:47:35.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "325.1"}]: dispatch 2024-02-05T07:47:35.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:34 smithi050 ceph-mon[33492]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "325.1"}]: dispatch 2024-02-05T07:47:35.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:34 smithi050 ceph-mon[33492]: 325.1 scrub starts 2024-02-05T07:47:35.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:34 smithi050 ceph-mon[33492]: 325.1 scrub ok 2024-02-05T07:47:36.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:35 smithi043 ceph-mon[29879]: pgmap v912: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-05T07:47:36.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:35 smithi043 ceph-mon[41444]: pgmap v912: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-05T07:47:36.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:35 smithi050 ceph-mon[33492]: pgmap v912: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-02-05T07:47:36.964 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:36 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:36.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:36 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:37.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:36 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:47:37.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:37 smithi043 ceph-mon[29879]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:37.980 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:37 smithi043 ceph-mon[41444]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:38.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:37 smithi050 ceph-mon[33492]: pgmap v913: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:47:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:47:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:47:40.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:39 smithi043 ceph-mon[29879]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 981 B/s rd, 1 op/s 2024-02-05T07:47:40.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:39 smithi043 ceph-mon[41444]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 981 B/s rd, 1 op/s 2024-02-05T07:47:40.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:39 smithi050 ceph-mon[33492]: pgmap v914: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 981 B/s rd, 1 op/s 2024-02-05T07:47:41.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:47:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:47:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:42 smithi043 ceph-mon[41444]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:47:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:42 smithi043 ceph-mon[29879]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:47:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:42 smithi050 ceph-mon[33492]: pgmap v915: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:47:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:44 smithi043 ceph-mon[29879]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:44 smithi043 ceph-mon[41444]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:44 smithi050 ceph-mon[33492]: pgmap v916: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:47:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:46 smithi043 ceph-mon[29879]: pgmap v917: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:47:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:46 smithi043 ceph-mon[41444]: pgmap v917: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:47:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:46 smithi050 ceph-mon[33492]: pgmap v917: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:47:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:48 smithi043 ceph-mon[29879]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:47:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:48 smithi043 ceph-mon[41444]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:47:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:48 smithi050 ceph-mon[33492]: pgmap v918: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-05T07:47:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:47:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:47:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:47:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[29879]: pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:47:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[29879]: osdmap e596: 8 total, 8 up, 8 in 2024-02-05T07:47:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[41444]: pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:47:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[41444]: osdmap e596: 8 total, 8 up, 8 in 2024-02-05T07:47:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:50 smithi050 ceph-mon[33492]: pgmap v919: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:47:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:50 smithi050 ceph-mon[33492]: osdmap e596: 8 total, 8 up, 8 in 2024-02-05T07:47:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[29879]: osdmap e597: 8 total, 8 up, 8 in 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]': finished 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[29879]: osdmap e598: 8 total, 8 up, 8 in 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[41444]: osdmap e597: 8 total, 8 up, 8 in 2024-02-05T07:47:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]': finished 2024-02-05T07:47:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:51 smithi043 ceph-mon[41444]: osdmap e598: 8 total, 8 up, 8 in 2024-02-05T07:47:51.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:47:51.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:47:51.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-mon[33492]: osdmap e597: 8 total, 8 up, 8 in 2024-02-05T07:47:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]: dispatch 2024-02-05T07:47:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:47:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-116"}]': finished 2024-02-05T07:47:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:51 smithi050 ceph-mon[33492]: osdmap e598: 8 total, 8 up, 8 in 2024-02-05T07:47:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:52 smithi043 ceph-mon[29879]: pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:47:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:52 smithi043 ceph-mon[41444]: pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:47:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:52 smithi050 ceph-mon[33492]: pgmap v922: 268 pgs: 268 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-05T07:47:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:53 smithi043 ceph-mon[41444]: osdmap e599: 8 total, 8 up, 8 in 2024-02-05T07:47:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:53 smithi043 ceph-mon[29879]: osdmap e599: 8 total, 8 up, 8 in 2024-02-05T07:47:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:47:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:53 smithi050 ceph-mon[33492]: osdmap e599: 8 total, 8 up, 8 in 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[41444]: pgmap v925: 236 pgs: 236 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[41444]: osdmap e600: 8 total, 8 up, 8 in 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[29879]: pgmap v925: 236 pgs: 236 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[29879]: osdmap e600: 8 total, 8 up, 8 in 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:54 smithi050 ceph-mon[33492]: pgmap v925: 236 pgs: 236 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-02-05T07:47:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:54 smithi050 ceph-mon[33492]: osdmap e600: 8 total, 8 up, 8 in 2024-02-05T07:47:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:47:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:55 smithi043 ceph-mon[29879]: osdmap e601: 8 total, 8 up, 8 in 2024-02-05T07:47:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:55 smithi043 ceph-mon[41444]: osdmap e601: 8 total, 8 up, 8 in 2024-02-05T07:47:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:47:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:55 smithi050 ceph-mon[33492]: osdmap e601: 8 total, 8 up, 8 in 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: osdmap e602: 8 total, 8 up, 8 in 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: osdmap e603: 8 total, 8 up, 8 in 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:47:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: osdmap e602: 8 total, 8 up, 8 in 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: osdmap e603: 8 total, 8 up, 8 in 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:47:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:47:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: pgmap v928: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:47:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: osdmap e602: 8 total, 8 up, 8 in 2024-02-05T07:47:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:47:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:47:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: osdmap e603: 8 total, 8 up, 8 in 2024-02-05T07:47:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:47:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:47:58.375 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:58 smithi050 ceph-mon[33492]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:58.375 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]': finished 2024-02-05T07:47:58.375 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:58 smithi050 ceph-mon[33492]: osdmap e604: 8 total, 8 up, 8 in 2024-02-05T07:47:58.375 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]: dispatch 2024-02-05T07:47:58.375 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]: dispatch 2024-02-05T07:47:58.375 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:58.375 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[29879]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]': finished 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[29879]: osdmap e604: 8 total, 8 up, 8 in 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]: dispatch 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]: dispatch 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[41444]: pgmap v931: 268 pgs: 268 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-118"}]': finished 2024-02-05T07:47:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[41444]: osdmap e604: 8 total, 8 up, 8 in 2024-02-05T07:47:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]: dispatch 2024-02-05T07:47:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]: dispatch 2024-02-05T07:47:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:47:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:47:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:59 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]': finished 2024-02-05T07:47:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:47:59 smithi043 ceph-mon[29879]: osdmap e605: 8 total, 8 up, 8 in 2024-02-05T07:47:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:47:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:47:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:47:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:59 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]': finished 2024-02-05T07:47:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:47:59 smithi043 ceph-mon[41444]: osdmap e605: 8 total, 8 up, 8 in 2024-02-05T07:47:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:59 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:47:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-118", "mode": "writeback"}]': finished 2024-02-05T07:47:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:47:59 smithi050 ceph-mon[33492]: osdmap e605: 8 total, 8 up, 8 in 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[29879]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[29879]: osdmap e606: 8 total, 8 up, 8 in 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[41444]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[41444]: osdmap e606: 8 total, 8 up, 8 in 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:00 smithi050 ceph-mon[33492]: pgmap v934: 268 pgs: 268 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:48:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:00 smithi050 ceph-mon[33492]: osdmap e606: 8 total, 8 up, 8 in 2024-02-05T07:48:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[29879]: osdmap e607: 8 total, 8 up, 8 in 2024-02-05T07:48:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:48:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:48:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]': finished 2024-02-05T07:48:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[41444]: osdmap e607: 8 total, 8 up, 8 in 2024-02-05T07:48:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:48:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:48:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]': finished 2024-02-05T07:48:01.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:48:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:48:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:01 smithi050 ceph-mon[33492]: osdmap e607: 8 total, 8 up, 8 in 2024-02-05T07:48:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:48:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]: dispatch 2024-02-05T07:48:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:01 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-118"}]': finished 2024-02-05T07:48:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:02 smithi043 ceph-mon[41444]: pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:48:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:02 smithi043 ceph-mon[41444]: osdmap e608: 8 total, 8 up, 8 in 2024-02-05T07:48:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:02 smithi043 ceph-mon[29879]: pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:48:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:02 smithi043 ceph-mon[29879]: osdmap e608: 8 total, 8 up, 8 in 2024-02-05T07:48:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:02 smithi050 ceph-mon[33492]: pgmap v937: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:48:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:02 smithi050 ceph-mon[33492]: osdmap e608: 8 total, 8 up, 8 in 2024-02-05T07:48:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:03 smithi043 ceph-mon[41444]: osdmap e609: 8 total, 8 up, 8 in 2024-02-05T07:48:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:03 smithi043 ceph-mon[29879]: osdmap e609: 8 total, 8 up, 8 in 2024-02-05T07:48:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:03 smithi050 ceph-mon[33492]: osdmap e609: 8 total, 8 up, 8 in 2024-02-05T07:48:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[29879]: pgmap v940: 236 pgs: 236 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[29879]: osdmap e610: 8 total, 8 up, 8 in 2024-02-05T07:48:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[41444]: pgmap v940: 236 pgs: 236 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[41444]: osdmap e610: 8 total, 8 up, 8 in 2024-02-05T07:48:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:04 smithi050 ceph-mon[33492]: pgmap v940: 236 pgs: 236 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:04 smithi050 ceph-mon[33492]: osdmap e610: 8 total, 8 up, 8 in 2024-02-05T07:48:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[41444]: osdmap e611: 8 total, 8 up, 8 in 2024-02-05T07:48:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[29879]: osdmap e611: 8 total, 8 up, 8 in 2024-02-05T07:48:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:05 smithi050 ceph-mon[33492]: osdmap e611: 8 total, 8 up, 8 in 2024-02-05T07:48:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:06.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: pgmap v943: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:06.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: osdmap e612: 8 total, 8 up, 8 in 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]': finished 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: osdmap e613: 8 total, 8 up, 8 in 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]: dispatch 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]: dispatch 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: pgmap v943: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: osdmap e612: 8 total, 8 up, 8 in 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]': finished 2024-02-05T07:48:06.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: osdmap e613: 8 total, 8 up, 8 in 2024-02-05T07:48:06.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]: dispatch 2024-02-05T07:48:06.466 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]: dispatch 2024-02-05T07:48:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: pgmap v943: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: osdmap e612: 8 total, 8 up, 8 in 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-120"}]': finished 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: osdmap e613: 8 total, 8 up, 8 in 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]: dispatch 2024-02-05T07:48:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]: dispatch 2024-02-05T07:48:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:07 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]': finished 2024-02-05T07:48:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:07 smithi043 ceph-mon[29879]: osdmap e614: 8 total, 8 up, 8 in 2024-02-05T07:48:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:07 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]': finished 2024-02-05T07:48:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:07 smithi043 ceph-mon[41444]: osdmap e614: 8 total, 8 up, 8 in 2024-02-05T07:48:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:07 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-120", "mode": "writeback"}]': finished 2024-02-05T07:48:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:07 smithi050 ceph-mon[33492]: osdmap e614: 8 total, 8 up, 8 in 2024-02-05T07:48:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:08 smithi043 ceph-mon[41444]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:08 smithi043 ceph-mon[29879]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:08 smithi050 ceph-mon[33492]: pgmap v946: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:48:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:48:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[29879]: osdmap e615: 8 total, 8 up, 8 in 2024-02-05T07:48:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:09.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:09.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[41444]: osdmap e615: 8 total, 8 up, 8 in 2024-02-05T07:48:09.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:09.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:09 smithi050 ceph-mon[33492]: osdmap e615: 8 total, 8 up, 8 in 2024-02-05T07:48:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]: dispatch 2024-02-05T07:48:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[29879]: pgmap v949: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]': finished 2024-02-05T07:48:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[29879]: osdmap e616: 8 total, 8 up, 8 in 2024-02-05T07:48:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[41444]: pgmap v949: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]': finished 2024-02-05T07:48:10.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:10 smithi043 ceph-mon[41444]: osdmap e616: 8 total, 8 up, 8 in 2024-02-05T07:48:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:10 smithi050 ceph-mon[33492]: pgmap v949: 268 pgs: 268 active+clean; 582 KiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:10 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-120"}]': finished 2024-02-05T07:48:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:10 smithi050 ceph-mon[33492]: osdmap e616: 8 total, 8 up, 8 in 2024-02-05T07:48:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:11 smithi043 ceph-mon[41444]: osdmap e617: 8 total, 8 up, 8 in 2024-02-05T07:48:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:11 smithi043 ceph-mon[29879]: osdmap e617: 8 total, 8 up, 8 in 2024-02-05T07:48:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:11 smithi050 ceph-mon[33492]: osdmap e617: 8 total, 8 up, 8 in 2024-02-05T07:48:11.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:48:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:48:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[41444]: pgmap v952: 236 pgs: 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[41444]: osdmap e618: 8 total, 8 up, 8 in 2024-02-05T07:48:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[29879]: pgmap v952: 236 pgs: 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[29879]: osdmap e618: 8 total, 8 up, 8 in 2024-02-05T07:48:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:12.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:12 smithi050 ceph-mon[33492]: pgmap v952: 236 pgs: 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:12 smithi050 ceph-mon[33492]: osdmap e618: 8 total, 8 up, 8 in 2024-02-05T07:48:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[29879]: osdmap e619: 8 total, 8 up, 8 in 2024-02-05T07:48:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[41444]: osdmap e619: 8 total, 8 up, 8 in 2024-02-05T07:48:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:13 smithi050 ceph-mon[33492]: osdmap e619: 8 total, 8 up, 8 in 2024-02-05T07:48:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[29879]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[29879]: osdmap e620: 8 total, 8 up, 8 in 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[41444]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[41444]: osdmap e620: 8 total, 8 up, 8 in 2024-02-05T07:48:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:14.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:14 smithi050 ceph-mon[33492]: pgmap v955: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:14 smithi050 ceph-mon[33492]: osdmap e620: 8 total, 8 up, 8 in 2024-02-05T07:48:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]': finished 2024-02-05T07:48:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[29879]: osdmap e621: 8 total, 8 up, 8 in 2024-02-05T07:48:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]: dispatch 2024-02-05T07:48:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]: dispatch 2024-02-05T07:48:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]': finished 2024-02-05T07:48:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[41444]: osdmap e621: 8 total, 8 up, 8 in 2024-02-05T07:48:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]: dispatch 2024-02-05T07:48:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]: dispatch 2024-02-05T07:48:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-122"}]': finished 2024-02-05T07:48:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:15 smithi050 ceph-mon[33492]: osdmap e621: 8 total, 8 up, 8 in 2024-02-05T07:48:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]: dispatch 2024-02-05T07:48:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]: dispatch 2024-02-05T07:48:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[29879]: pgmap v958: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]': finished 2024-02-05T07:48:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[29879]: osdmap e622: 8 total, 8 up, 8 in 2024-02-05T07:48:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[41444]: pgmap v958: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]': finished 2024-02-05T07:48:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[41444]: osdmap e622: 8 total, 8 up, 8 in 2024-02-05T07:48:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:16 smithi050 ceph-mon[33492]: pgmap v958: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:16 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-122", "mode": "writeback"}]': finished 2024-02-05T07:48:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:16 smithi050 ceph-mon[33492]: osdmap e622: 8 total, 8 up, 8 in 2024-02-05T07:48:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[29879]: osdmap e623: 8 total, 8 up, 8 in 2024-02-05T07:48:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:17.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[41444]: osdmap e623: 8 total, 8 up, 8 in 2024-02-05T07:48:17.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:17.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:17 smithi050 ceph-mon[33492]: osdmap e623: 8 total, 8 up, 8 in 2024-02-05T07:48:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:17.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]: dispatch 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[29879]: pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]': finished 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[29879]: osdmap e624: 8 total, 8 up, 8 in 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[41444]: pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]': finished 2024-02-05T07:48:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:18 smithi043 ceph-mon[41444]: osdmap e624: 8 total, 8 up, 8 in 2024-02-05T07:48:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:18 smithi050 ceph-mon[33492]: pgmap v961: 268 pgs: 268 active+clean; 582 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-05T07:48:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:18 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-122"}]': finished 2024-02-05T07:48:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:18 smithi050 ceph-mon[33492]: osdmap e624: 8 total, 8 up, 8 in 2024-02-05T07:48:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:48:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:48:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:48:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:19 smithi043 ceph-mon[29879]: osdmap e625: 8 total, 8 up, 8 in 2024-02-05T07:48:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:19 smithi043 ceph-mon[29879]: pgmap v964: 236 pgs: 236 active+clean; 582 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:48:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:19 smithi043 ceph-mon[41444]: osdmap e625: 8 total, 8 up, 8 in 2024-02-05T07:48:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:19 smithi043 ceph-mon[41444]: pgmap v964: 236 pgs: 236 active+clean; 582 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:48:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:19 smithi050 ceph-mon[33492]: osdmap e625: 8 total, 8 up, 8 in 2024-02-05T07:48:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:19 smithi050 ceph-mon[33492]: pgmap v964: 236 pgs: 236 active+clean; 582 KiB data, 671 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-05T07:48:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:20 smithi043 ceph-mon[29879]: osdmap e626: 8 total, 8 up, 8 in 2024-02-05T07:48:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:20 smithi043 ceph-mon[41444]: osdmap e626: 8 total, 8 up, 8 in 2024-02-05T07:48:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:20 smithi050 ceph-mon[33492]: osdmap e626: 8 total, 8 up, 8 in 2024-02-05T07:48:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[29879]: osdmap e627: 8 total, 8 up, 8 in 2024-02-05T07:48:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[29879]: pgmap v967: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[29879]: osdmap e628: 8 total, 8 up, 8 in 2024-02-05T07:48:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[41444]: osdmap e627: 8 total, 8 up, 8 in 2024-02-05T07:48:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[41444]: pgmap v967: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:21.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:21 smithi043 ceph-mon[41444]: osdmap e628: 8 total, 8 up, 8 in 2024-02-05T07:48:21.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:48:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:48:21.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:21 smithi050 ceph-mon[33492]: osdmap e627: 8 total, 8 up, 8 in 2024-02-05T07:48:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:21 smithi050 ceph-mon[33492]: pgmap v967: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:21 smithi050 ceph-mon[33492]: osdmap e628: 8 total, 8 up, 8 in 2024-02-05T07:48:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:22 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:22 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:22 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[29879]: osdmap e629: 8 total, 8 up, 8 in 2024-02-05T07:48:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[29879]: pgmap v970: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:48:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[41444]: osdmap e629: 8 total, 8 up, 8 in 2024-02-05T07:48:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:23.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:23 smithi043 ceph-mon[41444]: pgmap v970: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:48:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:23 smithi050 ceph-mon[33492]: osdmap e629: 8 total, 8 up, 8 in 2024-02-05T07:48:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:23 smithi050 ceph-mon[33492]: pgmap v970: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:48:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]': finished 2024-02-05T07:48:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[29879]: osdmap e630: 8 total, 8 up, 8 in 2024-02-05T07:48:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]: dispatch 2024-02-05T07:48:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]: dispatch 2024-02-05T07:48:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]': finished 2024-02-05T07:48:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[41444]: osdmap e630: 8 total, 8 up, 8 in 2024-02-05T07:48:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]: dispatch 2024-02-05T07:48:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]: dispatch 2024-02-05T07:48:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-124"}]': finished 2024-02-05T07:48:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:24 smithi050 ceph-mon[33492]: osdmap e630: 8 total, 8 up, 8 in 2024-02-05T07:48:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]: dispatch 2024-02-05T07:48:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]: dispatch 2024-02-05T07:48:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]': finished 2024-02-05T07:48:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[29879]: osdmap e631: 8 total, 8 up, 8 in 2024-02-05T07:48:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[29879]: pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]': finished 2024-02-05T07:48:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[41444]: osdmap e631: 8 total, 8 up, 8 in 2024-02-05T07:48:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:25 smithi043 ceph-mon[41444]: pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:25 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-124", "mode": "writeback"}]': finished 2024-02-05T07:48:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:25 smithi050 ceph-mon[33492]: osdmap e631: 8 total, 8 up, 8 in 2024-02-05T07:48:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:25 smithi050 ceph-mon[33492]: pgmap v973: 268 pgs: 268 active+clean; 582 KiB data, 672 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:26 smithi043 ceph-mon[29879]: osdmap e632: 8 total, 8 up, 8 in 2024-02-05T07:48:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:26 smithi043 ceph-mon[41444]: osdmap e632: 8 total, 8 up, 8 in 2024-02-05T07:48:26.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:26.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:26 smithi050 ceph-mon[33492]: osdmap e632: 8 total, 8 up, 8 in 2024-02-05T07:48:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: osdmap e633: 8 total, 8 up, 8 in 2024-02-05T07:48:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: pgmap v976: 268 pgs: 268 active+clean; 582 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-02-05T07:48:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]': finished 2024-02-05T07:48:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:27 smithi050 ceph-mon[33492]: osdmap e634: 8 total, 8 up, 8 in 2024-02-05T07:48:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: osdmap e633: 8 total, 8 up, 8 in 2024-02-05T07:48:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: pgmap v976: 268 pgs: 268 active+clean; 582 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-02-05T07:48:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]': finished 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[29879]: osdmap e634: 8 total, 8 up, 8 in 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: osdmap e633: 8 total, 8 up, 8 in 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]: dispatch 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: pgmap v976: 268 pgs: 268 active+clean; 582 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-124"}]': finished 2024-02-05T07:48:27.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:27 smithi043 ceph-mon[41444]: osdmap e634: 8 total, 8 up, 8 in 2024-02-05T07:48:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:48:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:48:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:48:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:29 smithi043 ceph-mon[29879]: osdmap e635: 8 total, 8 up, 8 in 2024-02-05T07:48:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:29 smithi043 ceph-mon[29879]: pgmap v979: 236 pgs: 236 active+clean; 582 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:48:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:29 smithi043 ceph-mon[41444]: osdmap e635: 8 total, 8 up, 8 in 2024-02-05T07:48:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:29 smithi043 ceph-mon[41444]: pgmap v979: 236 pgs: 236 active+clean; 582 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:48:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:29 smithi050 ceph-mon[33492]: osdmap e635: 8 total, 8 up, 8 in 2024-02-05T07:48:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:29 smithi050 ceph-mon[33492]: pgmap v979: 236 pgs: 236 active+clean; 582 KiB data, 676 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-05T07:48:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:48:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[29879]: osdmap e636: 8 total, 8 up, 8 in 2024-02-05T07:48:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:30.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[41444]: osdmap e636: 8 total, 8 up, 8 in 2024-02-05T07:48:30.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:30.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:30.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:30.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:30.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:30 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:30 smithi050 ceph-mon[33492]: osdmap e636: 8 total, 8 up, 8 in 2024-02-05T07:48:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:30 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:48:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:30 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:48:30.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:30 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:31.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: osdmap e637: 8 total, 8 up, 8 in 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: pgmap v982: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: osdmap e638: 8 total, 8 up, 8 in 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:31.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: osdmap e637: 8 total, 8 up, 8 in 2024-02-05T07:48:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: pgmap v982: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: osdmap e638: 8 total, 8 up, 8 in 2024-02-05T07:48:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:31.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:31.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:48:31.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: osdmap e637: 8 total, 8 up, 8 in 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: pgmap v982: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: osdmap e638: 8 total, 8 up, 8 in 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:31.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]': finished 2024-02-05T07:48:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[29879]: osdmap e639: 8 total, 8 up, 8 in 2024-02-05T07:48:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]: dispatch 2024-02-05T07:48:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]: dispatch 2024-02-05T07:48:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]': finished 2024-02-05T07:48:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[41444]: osdmap e639: 8 total, 8 up, 8 in 2024-02-05T07:48:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]: dispatch 2024-02-05T07:48:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]: dispatch 2024-02-05T07:48:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-126"}]': finished 2024-02-05T07:48:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:33 smithi050 ceph-mon[33492]: osdmap e639: 8 total, 8 up, 8 in 2024-02-05T07:48:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]: dispatch 2024-02-05T07:48:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]: dispatch 2024-02-05T07:48:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[29879]: pgmap v985: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]': finished 2024-02-05T07:48:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[29879]: osdmap e640: 8 total, 8 up, 8 in 2024-02-05T07:48:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[41444]: pgmap v985: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]': finished 2024-02-05T07:48:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[41444]: osdmap e640: 8 total, 8 up, 8 in 2024-02-05T07:48:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:34 smithi050 ceph-mon[33492]: pgmap v985: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:34 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-126", "mode": "writeback"}]': finished 2024-02-05T07:48:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:34 smithi050 ceph-mon[33492]: osdmap e640: 8 total, 8 up, 8 in 2024-02-05T07:48:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[41444]: osdmap e641: 8 total, 8 up, 8 in 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[29879]: osdmap e641: 8 total, 8 up, 8 in 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:35 smithi050 ceph-mon[33492]: osdmap e641: 8 total, 8 up, 8 in 2024-02-05T07:48:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]: dispatch 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sepi_tier_pp: require_osd_release = squid 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9054 ms) 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6038 ms) 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:48:36.151 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5022 ms) 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25662 ms) 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14046 ms) 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12062 ms) 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000012 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12051 ms) 2024-02-05T07:48:36.152 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6018 ms) 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (559351 ms total) 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1016 ms) 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11040 ms) 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18082 ms) 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30140 ms total) 2024-02-05T07:48:36.153 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7038 ms) 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8046 ms) 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrub... 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24597 ms) 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10006 ms) 2024-02-05T07:48:36.154 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-05T07:48:36.155 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8033 ms) 2024-02-05T07:48:36.155 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-05T07:48:36.155 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8050 ms) 2024-02-05T07:48:36.155 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-05T07:48:36.155 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10085 ms) 2024-02-05T07:48:36.155 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-05T07:48:36.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[41444]: pgmap v988: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:36.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:36.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]': finished 2024-02-05T07:48:36.464 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[41444]: osdmap e642: 8 total, 8 up, 8 in 2024-02-05T07:48:36.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[29879]: pgmap v988: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:36.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:36.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]': finished 2024-02-05T07:48:36.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:36 smithi043 ceph-mon[29879]: osdmap e642: 8 total, 8 up, 8 in 2024-02-05T07:48:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:36 smithi050 ceph-mon[33492]: pgmap v988: 268 pgs: 268 active+clean; 582 KiB data, 677 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:36 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-126"}]': finished 2024-02-05T07:48:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:36 smithi050 ceph-mon[33492]: osdmap e642: 8 total, 8 up, 8 in 2024-02-05T07:48:37.477 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:37 smithi043 ceph-mon[29879]: osdmap e643: 8 total, 8 up, 8 in 2024-02-05T07:48:37.477 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:37 smithi043 ceph-mon[41444]: osdmap e643: 8 total, 8 up, 8 in 2024-02-05T07:48:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:37 smithi050 ceph-mon[33492]: osdmap e643: 8 total, 8 up, 8 in 2024-02-05T07:48:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[41444]: pgmap v991: 236 pgs: 236 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[41444]: osdmap e644: 8 total, 8 up, 8 in 2024-02-05T07:48:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[29879]: pgmap v991: 236 pgs: 236 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[29879]: osdmap e644: 8 total, 8 up, 8 in 2024-02-05T07:48:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:38.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:38 smithi050 ceph-mon[33492]: pgmap v991: 236 pgs: 236 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:38 smithi050 ceph-mon[33492]: osdmap e644: 8 total, 8 up, 8 in 2024-02-05T07:48:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:48:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:48:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[29879]: osdmap e645: 8 total, 8 up, 8 in 2024-02-05T07:48:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[41444]: osdmap e645: 8 total, 8 up, 8 in 2024-02-05T07:48:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:39 smithi050 ceph-mon[33492]: osdmap e645: 8 total, 8 up, 8 in 2024-02-05T07:48:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[29879]: pgmap v994: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[29879]: osdmap e646: 8 total, 8 up, 8 in 2024-02-05T07:48:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[41444]: pgmap v994: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[41444]: osdmap e646: 8 total, 8 up, 8 in 2024-02-05T07:48:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:40 smithi050 ceph-mon[33492]: pgmap v994: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:40 smithi050 ceph-mon[33492]: osdmap e646: 8 total, 8 up, 8 in 2024-02-05T07:48:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]': finished 2024-02-05T07:48:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[29879]: osdmap e647: 8 total, 8 up, 8 in 2024-02-05T07:48:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]: dispatch 2024-02-05T07:48:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]: dispatch 2024-02-05T07:48:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]': finished 2024-02-05T07:48:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]': finished 2024-02-05T07:48:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[41444]: osdmap e647: 8 total, 8 up, 8 in 2024-02-05T07:48:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]: dispatch 2024-02-05T07:48:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]: dispatch 2024-02-05T07:48:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]': finished 2024-02-05T07:48:41.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:48:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:48:41.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-128"}]': finished 2024-02-05T07:48:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:41 smithi050 ceph-mon[33492]: osdmap e647: 8 total, 8 up, 8 in 2024-02-05T07:48:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]: dispatch 2024-02-05T07:48:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]: dispatch 2024-02-05T07:48:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:41 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-128", "mode": "writeback"}]': finished 2024-02-05T07:48:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:42 smithi043 ceph-mon[41444]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:42 smithi043 ceph-mon[41444]: osdmap e648: 8 total, 8 up, 8 in 2024-02-05T07:48:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:42 smithi043 ceph-mon[29879]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:42 smithi043 ceph-mon[29879]: osdmap e648: 8 total, 8 up, 8 in 2024-02-05T07:48:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:42 smithi050 ceph-mon[33492]: pgmap v997: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:48:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:42 smithi050 ceph-mon[33492]: osdmap e648: 8 total, 8 up, 8 in 2024-02-05T07:48:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:44.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:44 smithi043 ceph-mon[29879]: pgmap v999: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:48:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:44 smithi043 ceph-mon[41444]: pgmap v999: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:48:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:44 smithi050 ceph-mon[33492]: pgmap v999: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:48:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:46 smithi043 ceph-mon[29879]: pgmap v1000: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:48:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:46 smithi043 ceph-mon[41444]: pgmap v1000: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:48:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:46 smithi050 ceph-mon[33492]: pgmap v1000: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:48:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:47.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:47 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:47.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:47 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:47 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[29879]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 264 B/s wr, 1 op/s 2024-02-05T07:48:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[29879]: osdmap e649: 8 total, 8 up, 8 in 2024-02-05T07:48:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[41444]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 264 B/s wr, 1 op/s 2024-02-05T07:48:48.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:48.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[41444]: osdmap e649: 8 total, 8 up, 8 in 2024-02-05T07:48:48.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:48.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:48 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:48 smithi050 ceph-mon[33492]: pgmap v1001: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 264 B/s wr, 1 op/s 2024-02-05T07:48:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:48 smithi050 ceph-mon[33492]: osdmap e649: 8 total, 8 up, 8 in 2024-02-05T07:48:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:48 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:48.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:48 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]: dispatch 2024-02-05T07:48:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:48:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:48:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:48:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:49 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]': finished 2024-02-05T07:48:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:49 smithi043 ceph-mon[29879]: osdmap e650: 8 total, 8 up, 8 in 2024-02-05T07:48:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:49 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]': finished 2024-02-05T07:48:49.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:49 smithi043 ceph-mon[41444]: osdmap e650: 8 total, 8 up, 8 in 2024-02-05T07:48:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:49 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-128"}]': finished 2024-02-05T07:48:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:49 smithi050 ceph-mon[33492]: osdmap e650: 8 total, 8 up, 8 in 2024-02-05T07:48:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:50 smithi043 ceph-mon[29879]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 794 B/s rd, 264 B/s wr, 1 op/s 2024-02-05T07:48:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:50 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:48:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:50 smithi043 ceph-mon[29879]: osdmap e651: 8 total, 8 up, 8 in 2024-02-05T07:48:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:50 smithi043 ceph-mon[41444]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 794 B/s rd, 264 B/s wr, 1 op/s 2024-02-05T07:48:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:50 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:48:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:50 smithi043 ceph-mon[41444]: osdmap e651: 8 total, 8 up, 8 in 2024-02-05T07:48:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:50 smithi050 ceph-mon[33492]: pgmap v1004: 268 pgs: 268 active+clean; 582 KiB data, 678 MiB used, 715 GiB / 715 GiB avail; 794 B/s rd, 264 B/s wr, 1 op/s 2024-02-05T07:48:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:50 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:48:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:50 smithi050 ceph-mon[33492]: osdmap e651: 8 total, 8 up, 8 in 2024-02-05T07:48:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:51 smithi043 ceph-mon[29879]: osdmap e652: 8 total, 8 up, 8 in 2024-02-05T07:48:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:51 smithi043 ceph-mon[41444]: osdmap e652: 8 total, 8 up, 8 in 2024-02-05T07:48:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:51.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:48:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:48:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:51 smithi050 ceph-mon[33492]: osdmap e652: 8 total, 8 up, 8 in 2024-02-05T07:48:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:51.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[29879]: pgmap v1007: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[29879]: osdmap e653: 8 total, 8 up, 8 in 2024-02-05T07:48:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[41444]: pgmap v1007: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[41444]: osdmap e653: 8 total, 8 up, 8 in 2024-02-05T07:48:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:52.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:52 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:52 smithi050 ceph-mon[33492]: pgmap v1007: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:48:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:52 smithi050 ceph-mon[33492]: osdmap e653: 8 total, 8 up, 8 in 2024-02-05T07:48:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:52 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:52 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:48:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[29879]: osdmap e654: 8 total, 8 up, 8 in 2024-02-05T07:48:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]': finished 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[29879]: osdmap e655: 8 total, 8 up, 8 in 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[41444]: osdmap e654: 8 total, 8 up, 8 in 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]': finished 2024-02-05T07:48:53.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:53 smithi043 ceph-mon[41444]: osdmap e655: 8 total, 8 up, 8 in 2024-02-05T07:48:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:48:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:48:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:53 smithi050 ceph-mon[33492]: osdmap e654: 8 total, 8 up, 8 in 2024-02-05T07:48:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:53.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:53.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-130"}]': finished 2024-02-05T07:48:53.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:53 smithi050 ceph-mon[33492]: osdmap e655: 8 total, 8 up, 8 in 2024-02-05T07:48:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[29879]: pgmap v1010: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]: dispatch 2024-02-05T07:48:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]: dispatch 2024-02-05T07:48:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]': finished 2024-02-05T07:48:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[29879]: osdmap e656: 8 total, 8 up, 8 in 2024-02-05T07:48:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[41444]: pgmap v1010: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]: dispatch 2024-02-05T07:48:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]: dispatch 2024-02-05T07:48:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]': finished 2024-02-05T07:48:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:54 smithi043 ceph-mon[41444]: osdmap e656: 8 total, 8 up, 8 in 2024-02-05T07:48:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:54 smithi050 ceph-mon[33492]: pgmap v1010: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:48:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]: dispatch 2024-02-05T07:48:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]: dispatch 2024-02-05T07:48:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:54 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:48:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-130", "mode": "writeback"}]': finished 2024-02-05T07:48:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:54 smithi050 ceph-mon[33492]: osdmap e656: 8 total, 8 up, 8 in 2024-02-05T07:48:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:48:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[29879]: pgmap v1013: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[29879]: osdmap e657: 8 total, 8 up, 8 in 2024-02-05T07:48:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:48:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[41444]: pgmap v1013: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[41444]: osdmap e657: 8 total, 8 up, 8 in 2024-02-05T07:48:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:56 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:48:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:56 smithi050 ceph-mon[33492]: pgmap v1013: 268 pgs: 32 creating+peering, 236 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail 2024-02-05T07:48:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:48:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:56 smithi050 ceph-mon[33492]: osdmap e657: 8 total, 8 up, 8 in 2024-02-05T07:48:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]: dispatch 2024-02-05T07:48:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:56 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:48:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]': finished 2024-02-05T07:48:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[29879]: osdmap e658: 8 total, 8 up, 8 in 2024-02-05T07:48:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[29879]: pgmap v1016: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:48:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]': finished 2024-02-05T07:48:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[41444]: osdmap e658: 8 total, 8 up, 8 in 2024-02-05T07:48:57.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:57 smithi043 ceph-mon[41444]: pgmap v1016: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:48:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:57 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:48:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-130"}]': finished 2024-02-05T07:48:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:57 smithi050 ceph-mon[33492]: osdmap e658: 8 total, 8 up, 8 in 2024-02-05T07:48:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:57 smithi050 ceph-mon[33492]: pgmap v1016: 268 pgs: 268 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-02-05T07:48:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:58 smithi043 ceph-mon[29879]: osdmap e659: 8 total, 8 up, 8 in 2024-02-05T07:48:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:58 smithi043 ceph-mon[41444]: osdmap e659: 8 total, 8 up, 8 in 2024-02-05T07:48:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:58 smithi050 ceph-mon[33492]: osdmap e659: 8 total, 8 up, 8 in 2024-02-05T07:48:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:48:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:48:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:48:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:48:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[29879]: osdmap e660: 8 total, 8 up, 8 in 2024-02-05T07:48:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[29879]: pgmap v1019: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[41444]: osdmap e660: 8 total, 8 up, 8 in 2024-02-05T07:48:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:48:59 smithi043 ceph-mon[41444]: pgmap v1019: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:48:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:59 smithi050 ceph-mon[33492]: osdmap e660: 8 total, 8 up, 8 in 2024-02-05T07:48:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:48:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:48:59 smithi050 ceph-mon[33492]: pgmap v1019: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 697 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-05T07:49:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[29879]: osdmap e661: 8 total, 8 up, 8 in 2024-02-05T07:49:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[41444]: osdmap e661: 8 total, 8 up, 8 in 2024-02-05T07:49:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:00 smithi050 ceph-mon[33492]: osdmap e661: 8 total, 8 up, 8 in 2024-02-05T07:49:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[29879]: osdmap e662: 8 total, 8 up, 8 in 2024-02-05T07:49:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[29879]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:49:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[41444]: osdmap e662: 8 total, 8 up, 8 in 2024-02-05T07:49:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[41444]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:49:01.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:01 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:01.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:49:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:49:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:01.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:01 smithi050 ceph-mon[33492]: osdmap e662: 8 total, 8 up, 8 in 2024-02-05T07:49:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:01 smithi050 ceph-mon[33492]: pgmap v1022: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:49:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:01 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: osdmap e663: 8 total, 8 up, 8 in 2024-02-05T07:49:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]: dispatch 2024-02-05T07:49:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]: dispatch 2024-02-05T07:49:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]': finished 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[41444]: osdmap e664: 8 total, 8 up, 8 in 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: osdmap e663: 8 total, 8 up, 8 in 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]: dispatch 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]: dispatch 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:02.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]': finished 2024-02-05T07:49:02.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:02 smithi043 ceph-mon[29879]: osdmap e664: 8 total, 8 up, 8 in 2024-02-05T07:49:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: osdmap e663: 8 total, 8 up, 8 in 2024-02-05T07:49:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]: dispatch 2024-02-05T07:49:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]: dispatch 2024-02-05T07:49:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-132", "mode": "writeback"}]': finished 2024-02-05T07:49:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:02 smithi050 ceph-mon[33492]: osdmap e664: 8 total, 8 up, 8 in 2024-02-05T07:49:03.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:03 smithi043 ceph-mon[29879]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:03 smithi043 ceph-mon[41444]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:03 smithi050 ceph-mon[33492]: pgmap v1025: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:04 smithi043 ceph-mon[41444]: osdmap e665: 8 total, 8 up, 8 in 2024-02-05T07:49:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:04 smithi043 ceph-mon[29879]: osdmap e665: 8 total, 8 up, 8 in 2024-02-05T07:49:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:04 smithi050 ceph-mon[33492]: osdmap e665: 8 total, 8 up, 8 in 2024-02-05T07:49:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[29879]: osdmap e666: 8 total, 8 up, 8 in 2024-02-05T07:49:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[29879]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[41444]: osdmap e666: 8 total, 8 up, 8 in 2024-02-05T07:49:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:05 smithi043 ceph-mon[41444]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:05 smithi050 ceph-mon[33492]: osdmap e666: 8 total, 8 up, 8 in 2024-02-05T07:49:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:05 smithi050 ceph-mon[33492]: pgmap v1028: 268 pgs: 268 active+clean; 582 KiB data, 737 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[41444]: osdmap e667: 8 total, 8 up, 8 in 2024-02-05T07:49:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[29879]: osdmap e667: 8 total, 8 up, 8 in 2024-02-05T07:49:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:06 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:06 smithi050 ceph-mon[33492]: osdmap e667: 8 total, 8 up, 8 in 2024-02-05T07:49:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:06 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:07 smithi043 ceph-mon[41444]: osdmap e668: 8 total, 8 up, 8 in 2024-02-05T07:49:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:07 smithi043 ceph-mon[41444]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:49:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:07 smithi043 ceph-mon[29879]: osdmap e668: 8 total, 8 up, 8 in 2024-02-05T07:49:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:07 smithi043 ceph-mon[29879]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:49:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:07 smithi050 ceph-mon[33492]: osdmap e668: 8 total, 8 up, 8 in 2024-02-05T07:49:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:07 smithi050 ceph-mon[33492]: pgmap v1031: 268 pgs: 268 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:49:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:08 smithi050 ceph-mon[33492]: osdmap e669: 8 total, 8 up, 8 in 2024-02-05T07:49:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:08 smithi043 ceph-mon[29879]: osdmap e669: 8 total, 8 up, 8 in 2024-02-05T07:49:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:08 smithi043 ceph-mon[41444]: osdmap e669: 8 total, 8 up, 8 in 2024-02-05T07:49:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[41444]: osdmap e670: 8 total, 8 up, 8 in 2024-02-05T07:49:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[41444]: pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:49:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[29879]: osdmap e670: 8 total, 8 up, 8 in 2024-02-05T07:49:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-mon[29879]: pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:49:09.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:49:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:49:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:49:09.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:09 smithi050 ceph-mon[33492]: osdmap e670: 8 total, 8 up, 8 in 2024-02-05T07:49:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]: dispatch 2024-02-05T07:49:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:09 smithi050 ceph-mon[33492]: pgmap v1034: 268 pgs: 268 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-05T07:49:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:10 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:10 smithi043 ceph-mon[41444]: osdmap e671: 8 total, 8 up, 8 in 2024-02-05T07:49:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:10 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:10 smithi043 ceph-mon[29879]: osdmap e671: 8 total, 8 up, 8 in 2024-02-05T07:49:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:10 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-132"}]': finished 2024-02-05T07:49:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:10 smithi050 ceph-mon[33492]: osdmap e671: 8 total, 8 up, 8 in 2024-02-05T07:49:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:11 smithi043 ceph-mon[29879]: osdmap e672: 8 total, 8 up, 8 in 2024-02-05T07:49:11.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:11 smithi043 ceph-mon[29879]: pgmap v1037: 236 pgs: 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:49:11.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:11 smithi043 ceph-mon[41444]: osdmap e672: 8 total, 8 up, 8 in 2024-02-05T07:49:11.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:11 smithi043 ceph-mon[41444]: pgmap v1037: 236 pgs: 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:49:11.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:11 smithi050 ceph-mon[33492]: osdmap e672: 8 total, 8 up, 8 in 2024-02-05T07:49:11.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:11 smithi050 ceph-mon[33492]: pgmap v1037: 236 pgs: 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:49:11.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:49:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[29879]: osdmap e673: 8 total, 8 up, 8 in 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[41444]: osdmap e673: 8 total, 8 up, 8 in 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:12 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:12 smithi050 ceph-mon[33492]: osdmap e673: 8 total, 8 up, 8 in 2024-02-05T07:49:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:12 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[29879]: osdmap e674: 8 total, 8 up, 8 in 2024-02-05T07:49:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[29879]: pgmap v1040: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:49:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[41444]: osdmap e674: 8 total, 8 up, 8 in 2024-02-05T07:49:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[41444]: pgmap v1040: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:49:13.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:13 smithi050 ceph-mon[33492]: osdmap e674: 8 total, 8 up, 8 in 2024-02-05T07:49:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:13 smithi050 ceph-mon[33492]: pgmap v1040: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T07:49:13.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[29879]: osdmap e675: 8 total, 8 up, 8 in 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[41444]: osdmap e675: 8 total, 8 up, 8 in 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:14 smithi050 ceph-mon[33492]: osdmap e675: 8 total, 8 up, 8 in 2024-02-05T07:49:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]': finished 2024-02-05T07:49:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[29879]: osdmap e676: 8 total, 8 up, 8 in 2024-02-05T07:49:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]: dispatch 2024-02-05T07:49:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]: dispatch 2024-02-05T07:49:15.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[29879]: pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:15.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]': finished 2024-02-05T07:49:15.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[41444]: osdmap e676: 8 total, 8 up, 8 in 2024-02-05T07:49:15.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]: dispatch 2024-02-05T07:49:15.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]: dispatch 2024-02-05T07:49:15.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:15 smithi043 ceph-mon[41444]: pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-134"}]': finished 2024-02-05T07:49:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:15 smithi050 ceph-mon[33492]: osdmap e676: 8 total, 8 up, 8 in 2024-02-05T07:49:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]: dispatch 2024-02-05T07:49:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]: dispatch 2024-02-05T07:49:15.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:15 smithi050 ceph-mon[33492]: pgmap v1043: 268 pgs: 32 unknown, 236 active+clean; 582 KiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]': finished 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: osdmap e677: 8 total, 8 up, 8 in 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: osdmap e678: 8 total, 8 up, 8 in 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]': finished 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: osdmap e677: 8 total, 8 up, 8 in 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: osdmap e678: 8 total, 8 up, 8 in 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:16.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:16 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-134", "mode": "writeback"}]': finished 2024-02-05T07:49:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: osdmap e677: 8 total, 8 up, 8 in 2024-02-05T07:49:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:16.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:16.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: osdmap e678: 8 total, 8 up, 8 in 2024-02-05T07:49:16.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:16.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:16 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]: dispatch 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[41444]: pgmap v1046: 268 pgs: 268 active+clean; 582 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]': finished 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[41444]: osdmap e679: 8 total, 8 up, 8 in 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[29879]: pgmap v1046: 268 pgs: 268 active+clean; 582 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]': finished 2024-02-05T07:49:17.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:17 smithi043 ceph-mon[29879]: osdmap e679: 8 total, 8 up, 8 in 2024-02-05T07:49:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:17 smithi050 ceph-mon[33492]: pgmap v1046: 268 pgs: 268 active+clean; 582 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-05T07:49:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:17 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-134"}]': finished 2024-02-05T07:49:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:17 smithi050 ceph-mon[33492]: osdmap e679: 8 total, 8 up, 8 in 2024-02-05T07:49:19.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:19 smithi043 ceph-mon[41444]: osdmap e680: 8 total, 8 up, 8 in 2024-02-05T07:49:19.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:19 smithi043 ceph-mon[29879]: osdmap e680: 8 total, 8 up, 8 in 2024-02-05T07:49:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:49:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:49:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:49:19.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:19 smithi050 ceph-mon[33492]: osdmap e680: 8 total, 8 up, 8 in 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[41444]: pgmap v1049: 236 pgs: 236 active+clean; 582 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[41444]: osdmap e681: 8 total, 8 up, 8 in 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[29879]: pgmap v1049: 236 pgs: 236 active+clean; 582 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[29879]: osdmap e681: 8 total, 8 up, 8 in 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:20 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:20 smithi050 ceph-mon[33492]: pgmap v1049: 236 pgs: 236 active+clean; 582 KiB data, 739 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T07:49:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:20 smithi050 ceph-mon[33492]: osdmap e681: 8 total, 8 up, 8 in 2024-02-05T07:49:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:20 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:20 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[41444]: osdmap e682: 8 total, 8 up, 8 in 2024-02-05T07:49:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:21.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[29879]: osdmap e682: 8 total, 8 up, 8 in 2024-02-05T07:49:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:21.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:21 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:21.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:21.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:21 smithi050 ceph-mon[33492]: osdmap e682: 8 total, 8 up, 8 in 2024-02-05T07:49:21.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:21 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:21.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:21 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:21.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:49:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[41444]: pgmap v1052: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[41444]: osdmap e683: 8 total, 8 up, 8 in 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[29879]: pgmap v1052: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[29879]: osdmap e683: 8 total, 8 up, 8 in 2024-02-05T07:49:22.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:22.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:22 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:22 smithi050 ceph-mon[33492]: pgmap v1052: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:22 smithi050 ceph-mon[33492]: osdmap e683: 8 total, 8 up, 8 in 2024-02-05T07:49:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:22 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:22 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:23.503 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] c 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.504 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.505 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.506 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.507 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.508 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.509 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.510 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:49:23.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]': finished 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[29879]: osdmap e684: 8 total, 8 up, 8 in 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]: dispatch 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]: dispatch 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]': finished 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[41444]: osdmap e684: 8 total, 8 up, 8 in 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]: dispatch 2024-02-05T07:49:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]: dispatch 2024-02-05T07:49:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-136"}]': finished 2024-02-05T07:49:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:23 smithi050 ceph-mon[33492]: osdmap e684: 8 total, 8 up, 8 in 2024-02-05T07:49:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]: dispatch 2024-02-05T07:49:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]: dispatch 2024-02-05T07:49:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[29879]: pgmap v1055: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]': finished 2024-02-05T07:49:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[29879]: osdmap e685: 8 total, 8 up, 8 in 2024-02-05T07:49:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[41444]: pgmap v1055: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]': finished 2024-02-05T07:49:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[41444]: osdmap e685: 8 total, 8 up, 8 in 2024-02-05T07:49:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:24.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:24 smithi050 ceph-mon[33492]: pgmap v1055: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:24 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:49:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-136", "mode": "writeback"}]': finished 2024-02-05T07:49:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:24 smithi050 ceph-mon[33492]: osdmap e685: 8 total, 8 up, 8 in 2024-02-05T07:49:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[29879]: osdmap e686: 8 total, 8 up, 8 in 2024-02-05T07:49:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:25.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:25.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[41444]: osdmap e686: 8 total, 8 up, 8 in 2024-02-05T07:49:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:25.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:49:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:25 smithi050 ceph-mon[33492]: osdmap e686: 8 total, 8 up, 8 in 2024-02-05T07:49:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:25.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]: dispatch 2024-02-05T07:49:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[29879]: pgmap v1058: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]': finished 2024-02-05T07:49:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[29879]: osdmap e687: 8 total, 8 up, 8 in 2024-02-05T07:49:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[41444]: pgmap v1058: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]': finished 2024-02-05T07:49:26.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:26 smithi043 ceph-mon[41444]: osdmap e687: 8 total, 8 up, 8 in 2024-02-05T07:49:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:26 smithi050 ceph-mon[33492]: pgmap v1058: 268 pgs: 268 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:49:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:26 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:49:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-136"}]': finished 2024-02-05T07:49:26.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:26 smithi050 ceph-mon[33492]: osdmap e687: 8 total, 8 up, 8 in 2024-02-05T07:49:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:27 smithi043 ceph-mon[29879]: osdmap e688: 8 total, 8 up, 8 in 2024-02-05T07:49:27.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:27 smithi043 ceph-mon[41444]: osdmap e688: 8 total, 8 up, 8 in 2024-02-05T07:49:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:27 smithi050 ceph-mon[33492]: osdmap e688: 8 total, 8 up, 8 in 2024-02-05T07:49:28.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:28 smithi050 ceph-mon[33492]: pgmap v1061: 236 pgs: 236 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:49:28.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:28 smithi050 ceph-mon[33492]: osdmap e689: 8 total, 8 up, 8 in 2024-02-05T07:49:28.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:28.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:28.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[29879]: pgmap v1061: 236 pgs: 236 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:49:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[29879]: osdmap e689: 8 total, 8 up, 8 in 2024-02-05T07:49:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[41444]: pgmap v1061: 236 pgs: 236 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:49:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[41444]: osdmap e689: 8 total, 8 up, 8 in 2024-02-05T07:49:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:28.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[29879]: osdmap e690: 8 total, 8 up, 8 in 2024-02-05T07:49:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:49:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:49:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[41444]: osdmap e690: 8 total, 8 up, 8 in 2024-02-05T07:49:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:29 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:29 smithi050 ceph-mon[33492]: osdmap e690: 8 total, 8 up, 8 in 2024-02-05T07:49:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:29 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:29 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[41444]: pgmap v1064: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[41444]: osdmap e691: 8 total, 8 up, 8 in 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[29879]: pgmap v1064: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[29879]: osdmap e691: 8 total, 8 up, 8 in 2024-02-05T07:49:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:49:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:49:30.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:30 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:30 smithi050 ceph-mon[33492]: pgmap v1064: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 754 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-02-05T07:49:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:30 smithi050 ceph-mon[33492]: osdmap e691: 8 total, 8 up, 8 in 2024-02-05T07:49:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:49:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:49:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:30 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: osdmap e692: 8 total, 8 up, 8 in 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: osdmap e692: 8 total, 8 up, 8 in 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:31 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:49:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:49:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: osdmap e692: 8 total, 8 up, 8 in 2024-02-05T07:49:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:49:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:49:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:49:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:49:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:31 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[41444]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[41444]: osdmap e693: 8 total, 8 up, 8 in 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[29879]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:49:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[29879]: osdmap e693: 8 total, 8 up, 8 in 2024-02-05T07:49:32.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:49:32.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:49:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:32 smithi050 ceph-mon[33492]: pgmap v1067: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:49:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:32 smithi050 ceph-mon[33492]: osdmap e693: 8 total, 8 up, 8 in 2024-02-05T07:49:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:49:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-05T07:49:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:49:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:33 smithi043 ceph-mon[41444]: osdmap e694: 8 total, 8 up, 8 in 2024-02-05T07:49:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:49:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:33 smithi043 ceph-mon[29879]: osdmap e694: 8 total, 8 up, 8 in 2024-02-05T07:49:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-05T07:49:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:33 smithi050 ceph-mon[33492]: osdmap e694: 8 total, 8 up, 8 in 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[41444]: pgmap v1070: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]: dispatch 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]: dispatch 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[29879]: pgmap v1070: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]: dispatch 2024-02-05T07:49:34.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]: dispatch 2024-02-05T07:49:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:34 smithi050 ceph-mon[33492]: pgmap v1070: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]: dispatch 2024-02-05T07:49:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]: dispatch 2024-02-05T07:49:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]': finished 2024-02-05T07:49:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:35 smithi043 ceph-mon[29879]: osdmap e695: 8 total, 8 up, 8 in 2024-02-05T07:49:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]': finished 2024-02-05T07:49:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:35 smithi043 ceph-mon[41444]: osdmap e695: 8 total, 8 up, 8 in 2024-02-05T07:49:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-138"}]': finished 2024-02-05T07:49:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:35 smithi050 ceph-mon[33492]: osdmap e695: 8 total, 8 up, 8 in 2024-02-05T07:49:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:36 smithi043 ceph-mon[29879]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:49:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:36 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:36 smithi043 ceph-mon[29879]: osdmap e696: 8 total, 8 up, 8 in 2024-02-05T07:49:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:36 smithi043 ceph-mon[41444]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:49:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:36 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:36 smithi043 ceph-mon[41444]: osdmap e696: 8 total, 8 up, 8 in 2024-02-05T07:49:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:36 smithi050 ceph-mon[33492]: pgmap v1072: 268 pgs: 268 active+clean; 4.4 MiB data, 777 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:49:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:36 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:36 smithi050 ceph-mon[33492]: osdmap e696: 8 total, 8 up, 8 in 2024-02-05T07:49:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:37 smithi043 ceph-mon[41444]: osdmap e697: 8 total, 8 up, 8 in 2024-02-05T07:49:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:37.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:37.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:37 smithi043 ceph-mon[29879]: osdmap e697: 8 total, 8 up, 8 in 2024-02-05T07:49:37.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:37.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:37 smithi050 ceph-mon[33492]: osdmap e697: 8 total, 8 up, 8 in 2024-02-05T07:49:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:37.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[41444]: pgmap v1075: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[41444]: osdmap e698: 8 total, 8 up, 8 in 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[29879]: pgmap v1075: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[29879]: osdmap e698: 8 total, 8 up, 8 in 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:38 smithi050 ceph-mon[33492]: pgmap v1075: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-05T07:49:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:38 smithi050 ceph-mon[33492]: osdmap e698: 8 total, 8 up, 8 in 2024-02-05T07:49:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:49:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:49:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[41444]: osdmap e699: 8 total, 8 up, 8 in 2024-02-05T07:49:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:49:39.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[41444]: pgmap v1078: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[41444]: osdmap e700: 8 total, 8 up, 8 in 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[29879]: osdmap e699: 8 total, 8 up, 8 in 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[29879]: pgmap v1078: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:49:39.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:39 smithi043 ceph-mon[29879]: osdmap e700: 8 total, 8 up, 8 in 2024-02-05T07:49:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:49:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:39 smithi050 ceph-mon[33492]: osdmap e699: 8 total, 8 up, 8 in 2024-02-05T07:49:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:49:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]: dispatch 2024-02-05T07:49:39.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:39 smithi050 ceph-mon[33492]: pgmap v1078: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 781 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_count","val": "3"}]': finished 2024-02-05T07:49:39.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:39 smithi050 ceph-mon[33492]: osdmap e700: 8 total, 8 up, 8 in 2024-02-05T07:49:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:49:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:49:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[41444]: osdmap e701: 8 total, 8 up, 8 in 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[29879]: osdmap e701: 8 total, 8 up, 8 in 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:49:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:49:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:49:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]: dispatch 2024-02-05T07:49:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_period","val": "3"}]': finished 2024-02-05T07:49:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:40 smithi050 ceph-mon[33492]: osdmap e701: 8 total, 8 up, 8 in 2024-02-05T07:49:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:49:40.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:49:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[29879]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:49:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[29879]: osdmap e702: 8 total, 8 up, 8 in 2024-02-05T07:49:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:49:41.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:49:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[41444]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:49:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[41444]: osdmap e702: 8 total, 8 up, 8 in 2024-02-05T07:49:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:49:41.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:49:41.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:49:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:49:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:41 smithi050 ceph-mon[33492]: pgmap v1081: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:41 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:49:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:49:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:41 smithi050 ceph-mon[33492]: osdmap e702: 8 total, 8 up, 8 in 2024-02-05T07:49:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:49:41.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-05T07:49:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:42 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:49:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:42 smithi043 ceph-mon[41444]: osdmap e703: 8 total, 8 up, 8 in 2024-02-05T07:49:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:42 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:49:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:42 smithi043 ceph-mon[29879]: osdmap e703: 8 total, 8 up, 8 in 2024-02-05T07:49:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:42 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-05T07:49:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:42 smithi050 ceph-mon[33492]: osdmap e703: 8 total, 8 up, 8 in 2024-02-05T07:49:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:43 smithi043 ceph-mon[29879]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:43 smithi043 ceph-mon[41444]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:43 smithi050 ceph-mon[33492]: pgmap v1084: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:46.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:45 smithi043 ceph-mon[29879]: pgmap v1085: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-02-05T07:49:46.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:45 smithi043 ceph-mon[41444]: pgmap v1085: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-02-05T07:49:46.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:45 smithi050 ceph-mon[33492]: pgmap v1085: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 904 B/s rd, 0 op/s 2024-02-05T07:49:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:48 smithi043 ceph-mon[29879]: pgmap v1086: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 2 op/s 2024-02-05T07:49:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:48 smithi043 ceph-mon[41444]: pgmap v1086: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 2 op/s 2024-02-05T07:49:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:48 smithi050 ceph-mon[33492]: pgmap v1086: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 5.0 KiB/s wr, 2 op/s 2024-02-05T07:49:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:49:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:49:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:49:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:50 smithi043 ceph-mon[29879]: pgmap v1087: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T07:49:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:50 smithi043 ceph-mon[41444]: pgmap v1087: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T07:49:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:50 smithi050 ceph-mon[33492]: pgmap v1087: 268 pgs: 268 active+clean; 4.4 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T07:49:51.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:49:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:49:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:52 smithi043 ceph-mon[29879]: pgmap v1088: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-02-05T07:49:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:52 smithi043 ceph-mon[41444]: pgmap v1088: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-02-05T07:49:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:52 smithi050 ceph-mon[33492]: pgmap v1088: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 7.7 KiB/s wr, 2 op/s 2024-02-05T07:49:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:49:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:54 smithi043 ceph-mon[29879]: pgmap v1089: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 960 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-02-05T07:49:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:54 smithi043 ceph-mon[41444]: pgmap v1089: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 960 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-02-05T07:49:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:54 smithi050 ceph-mon[33492]: pgmap v1089: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 960 B/s rd, 6.9 KiB/s wr, 1 op/s 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]: dispatch 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]: dispatch 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]: dispatch 2024-02-05T07:49:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]: dispatch 2024-02-05T07:49:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:49:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]: dispatch 2024-02-05T07:49:55.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]: dispatch 2024-02-05T07:49:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:56 smithi043 ceph-mon[29879]: pgmap v1090: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-05T07:49:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]': finished 2024-02-05T07:49:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:56 smithi043 ceph-mon[29879]: osdmap e704: 8 total, 8 up, 8 in 2024-02-05T07:49:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:56 smithi043 ceph-mon[41444]: pgmap v1090: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-05T07:49:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]': finished 2024-02-05T07:49:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:56 smithi043 ceph-mon[41444]: osdmap e704: 8 total, 8 up, 8 in 2024-02-05T07:49:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:56 smithi050 ceph-mon[33492]: pgmap v1090: 268 pgs: 268 active+clean; 4.5 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-05T07:49:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-140"}]': finished 2024-02-05T07:49:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:56 smithi050 ceph-mon[33492]: osdmap e704: 8 total, 8 up, 8 in 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[29879]: osdmap e705: 8 total, 8 up, 8 in 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[29879]: pgmap v1093: 236 pgs: 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[29879]: osdmap e706: 8 total, 8 up, 8 in 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[41444]: osdmap e705: 8 total, 8 up, 8 in 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[41444]: pgmap v1093: 236 pgs: 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[41444]: osdmap e706: 8 total, 8 up, 8 in 2024-02-05T07:49:57.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:57.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:57 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:57 smithi050 ceph-mon[33492]: osdmap e705: 8 total, 8 up, 8 in 2024-02-05T07:49:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:57 smithi050 ceph-mon[33492]: pgmap v1093: 236 pgs: 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:49:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:57 smithi050 ceph-mon[33492]: osdmap e706: 8 total, 8 up, 8 in 2024-02-05T07:49:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:57 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:57 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:49:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:58 smithi050 ceph-mon[33492]: osdmap e707: 8 total, 8 up, 8 in 2024-02-05T07:49:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:49:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[29879]: osdmap e707: 8 total, 8 up, 8 in 2024-02-05T07:49:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:49:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:49:58.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[41444]: osdmap e707: 8 total, 8 up, 8 in 2024-02-05T07:49:58.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:58.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:49:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:49:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:49:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:49:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: pgmap v1096: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: osdmap e708: 8 total, 8 up, 8 in 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T07:50:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: application not enabled on pool 'test-rados-api-smithi043-81993-111' 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[29879]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: pgmap v1096: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: osdmap e708: 8 total, 8 up, 8 in 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T07:50:00.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T07:50:00.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: application not enabled on pool 'test-rados-api-smithi043-81993-111' 2024-02-05T07:50:00.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:00 smithi043 ceph-mon[41444]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T07:50:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: pgmap v1096: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 784 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: osdmap e708: 8 total, 8 up, 8 in 2024-02-05T07:50:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: Health detail: HEALTH_WARN 4 pool(s) do not have an application enabled 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: [WRN] POOL_APP_NOT_ENABLED: 4 pool(s) do not have an application enabled 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: application not enabled on pool 'test-rados-api-smithi043-81993-111' 2024-02-05T07:50:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:00 smithi050 ceph-mon[33492]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]': finished 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: osdmap e709: 8 total, 8 up, 8 in 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]: dispatch 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]: dispatch 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]': finished 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: osdmap e710: 8 total, 8 up, 8 in 2024-02-05T07:50:01.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]': finished 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: osdmap e709: 8 total, 8 up, 8 in 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]: dispatch 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]: dispatch 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]': finished 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: osdmap e710: 8 total, 8 up, 8 in 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-142"}]': finished 2024-02-05T07:50:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: osdmap e709: 8 total, 8 up, 8 in 2024-02-05T07:50:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]: dispatch 2024-02-05T07:50:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]: dispatch 2024-02-05T07:50:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-142", "mode": "writeback"}]': finished 2024-02-05T07:50:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: osdmap e710: 8 total, 8 up, 8 in 2024-02-05T07:50:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:01.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:50:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:50:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[29879]: pgmap v1099: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:50:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:50:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[29879]: osdmap e711: 8 total, 8 up, 8 in 2024-02-05T07:50:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[41444]: pgmap v1099: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:50:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:50:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:02 smithi043 ceph-mon[41444]: osdmap e711: 8 total, 8 up, 8 in 2024-02-05T07:50:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:02 smithi050 ceph-mon[33492]: pgmap v1099: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:50:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:50:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:02 smithi050 ceph-mon[33492]: osdmap e711: 8 total, 8 up, 8 in 2024-02-05T07:50:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:50:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[29879]: osdmap e712: 8 total, 8 up, 8 in 2024-02-05T07:50:03.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:03.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:03.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:03.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:50:03.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:03 smithi043 ceph-mon[41444]: osdmap e712: 8 total, 8 up, 8 in 2024-02-05T07:50:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:03.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:50:03.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:03 smithi050 ceph-mon[33492]: osdmap e712: 8 total, 8 up, 8 in 2024-02-05T07:50:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[29879]: pgmap v1102: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:50:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:50:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[29879]: osdmap e713: 8 total, 8 up, 8 in 2024-02-05T07:50:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[41444]: pgmap v1102: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:50:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:50:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:04 smithi043 ceph-mon[41444]: osdmap e713: 8 total, 8 up, 8 in 2024-02-05T07:50:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:04 smithi050 ceph-mon[33492]: pgmap v1102: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-05T07:50:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:04 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:50:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:04 smithi050 ceph-mon[33492]: osdmap e713: 8 total, 8 up, 8 in 2024-02-05T07:50:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:50:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[29879]: osdmap e714: 8 total, 8 up, 8 in 2024-02-05T07:50:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:05.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:50:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:05 smithi043 ceph-mon[41444]: osdmap e714: 8 total, 8 up, 8 in 2024-02-05T07:50:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:50:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:05 smithi050 ceph-mon[33492]: osdmap e714: 8 total, 8 up, 8 in 2024-02-05T07:50:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[29879]: pgmap v1105: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:50:06.464 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[29879]: osdmap e715: 8 total, 8 up, 8 in 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[41444]: pgmap v1105: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[41444]: osdmap e715: 8 total, 8 up, 8 in 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:50:06.465 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:06 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:50:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:06 smithi050 ceph-mon[33492]: pgmap v1105: 268 pgs: 268 active+clean; 4.4 MiB data, 806 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:50:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-05T07:50:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-05T07:50:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:06 smithi050 ceph-mon[33492]: osdmap e715: 8 total, 8 up, 8 in 2024-02-05T07:50:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:06 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:50:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:06 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-05T07:50:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[41444]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:50:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[41444]: osdmap e716: 8 total, 8 up, 8 in 2024-02-05T07:50:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[29879]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:50:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[29879]: osdmap e716: 8 total, 8 up, 8 in 2024-02-05T07:50:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:08.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:08 smithi050 ceph-mon[33492]: pgmap v1108: 268 pgs: 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-05T07:50:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:08 smithi050 ceph-mon[33492]: osdmap e716: 8 total, 8 up, 8 in 2024-02-05T07:50:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[41444]: osdmap e717: 8 total, 8 up, 8 in 2024-02-05T07:50:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[29879]: osdmap e717: 8 total, 8 up, 8 in 2024-02-05T07:50:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:09.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:09.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:50:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:50:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:09 smithi050 ceph-mon[33492]: osdmap e717: 8 total, 8 up, 8 in 2024-02-05T07:50:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:09 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[41444]: pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]': finished 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[41444]: osdmap e718: 8 total, 8 up, 8 in 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[29879]: pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]': finished 2024-02-05T07:50:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[29879]: osdmap e718: 8 total, 8 up, 8 in 2024-02-05T07:50:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:10.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:10 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:10 smithi050 ceph-mon[33492]: pgmap v1111: 268 pgs: 268 active+clean; 4.4 MiB data, 807 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:50:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]': finished 2024-02-05T07:50:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:10 smithi050 ceph-mon[33492]: osdmap e718: 8 total, 8 up, 8 in 2024-02-05T07:50:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:10 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:10 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-142"}]: dispatch 2024-02-05T07:50:11.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:11 smithi043 ceph-mon[29879]: osdmap e719: 8 total, 8 up, 8 in 2024-02-05T07:50:11.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:11 smithi043 ceph-mon[41444]: osdmap e719: 8 total, 8 up, 8 in 2024-02-05T07:50:11.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:11 smithi050 ceph-mon[33492]: osdmap e719: 8 total, 8 up, 8 in 2024-02-05T07:50:11.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:50:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[29879]: pgmap v1114: 236 pgs: 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[29879]: osdmap e720: 8 total, 8 up, 8 in 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[41444]: pgmap v1114: 236 pgs: 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[41444]: osdmap e720: 8 total, 8 up, 8 in 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:12 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:12 smithi050 ceph-mon[33492]: pgmap v1114: 236 pgs: 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:50:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:12 smithi050 ceph-mon[33492]: osdmap e720: 8 total, 8 up, 8 in 2024-02-05T07:50:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:12 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:12 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[29879]: osdmap e721: 8 total, 8 up, 8 in 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[41444]: osdmap e721: 8 total, 8 up, 8 in 2024-02-05T07:50:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:13.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:13 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:13 smithi050 ceph-mon[33492]: osdmap e721: 8 total, 8 up, 8 in 2024-02-05T07:50:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:13 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:13 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:13 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[29879]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[29879]: osdmap e722: 8 total, 8 up, 8 in 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[41444]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[41444]: osdmap e722: 8 total, 8 up, 8 in 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:14.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:14 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:14 smithi050 ceph-mon[33492]: pgmap v1117: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-05T07:50:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:14 smithi050 ceph-mon[33492]: osdmap e722: 8 total, 8 up, 8 in 2024-02-05T07:50:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:14 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:14 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]': finished 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[29879]: osdmap e723: 8 total, 8 up, 8 in 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]: dispatch 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]: dispatch 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]': finished 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[29879]: osdmap e724: 8 total, 8 up, 8 in 2024-02-05T07:50:15.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]': finished 2024-02-05T07:50:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[41444]: osdmap e723: 8 total, 8 up, 8 in 2024-02-05T07:50:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]: dispatch 2024-02-05T07:50:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]: dispatch 2024-02-05T07:50:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]': finished 2024-02-05T07:50:15.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:15 smithi043 ceph-mon[41444]: osdmap e724: 8 total, 8 up, 8 in 2024-02-05T07:50:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-144"}]': finished 2024-02-05T07:50:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:15 smithi050 ceph-mon[33492]: osdmap e723: 8 total, 8 up, 8 in 2024-02-05T07:50:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:15 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]: dispatch 2024-02-05T07:50:15.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]: dispatch 2024-02-05T07:50:15.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:15 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:15.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:15 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-144", "mode": "readproxy"}]': finished 2024-02-05T07:50:15.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:15 smithi050 ceph-mon[33492]: osdmap e724: 8 total, 8 up, 8 in 2024-02-05T07:50:16.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:16 smithi043 ceph-mon[29879]: pgmap v1120: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:16 smithi043 ceph-mon[41444]: pgmap v1120: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:16 smithi050 ceph-mon[33492]: pgmap v1120: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 791 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:18 smithi043 ceph-mon[29879]: pgmap v1122: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 217 B/s wr, 1 op/s 2024-02-05T07:50:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:18 smithi043 ceph-mon[41444]: pgmap v1122: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 217 B/s wr, 1 op/s 2024-02-05T07:50:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:18 smithi050 ceph-mon[33492]: pgmap v1122: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 217 B/s wr, 1 op/s 2024-02-05T07:50:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:50:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:50:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:20 smithi043 ceph-mon[29879]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-05T07:50:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:20 smithi043 ceph-mon[41444]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-05T07:50:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:20 smithi050 ceph-mon[33492]: pgmap v1123: 268 pgs: 268 active+clean; 4.4 MiB data, 813 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s 2024-02-05T07:50:21.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:50:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:50:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:22 smithi043 ceph-mon[29879]: pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 132 B/s wr, 1 op/s 2024-02-05T07:50:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:22 smithi043 ceph-mon[41444]: pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 132 B/s wr, 1 op/s 2024-02-05T07:50:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:22 smithi050 ceph-mon[33492]: pgmap v1124: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 132 B/s wr, 1 op/s 2024-02-05T07:50:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:24 smithi043 ceph-mon[29879]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 117 B/s wr, 1 op/s 2024-02-05T07:50:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:24 smithi043 ceph-mon[41444]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 117 B/s wr, 1 op/s 2024-02-05T07:50:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:24 smithi050 ceph-mon[33492]: pgmap v1125: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 117 B/s wr, 1 op/s 2024-02-05T07:50:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:26 smithi043 ceph-mon[41444]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-05T07:50:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:26 smithi043 ceph-mon[29879]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-05T07:50:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:26.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:26 smithi050 ceph-mon[33492]: pgmap v1126: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 102 B/s wr, 1 op/s 2024-02-05T07:50:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[29879]: osdmap e725: 8 total, 8 up, 8 in 2024-02-05T07:50:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:27.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[29879]: pgmap v1128: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:50:27.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:27.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[41444]: osdmap e725: 8 total, 8 up, 8 in 2024-02-05T07:50:27.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:27.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:27.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:27 smithi043 ceph-mon[41444]: pgmap v1128: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:50:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:27.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:27 smithi050 ceph-mon[33492]: osdmap e725: 8 total, 8 up, 8 in 2024-02-05T07:50:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:27.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:27 smithi050 ceph-mon[33492]: pgmap v1128: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]': finished 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: osdmap e726: 8 total, 8 up, 8 in 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]': finished 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: osdmap e726: 8 total, 8 up, 8 in 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:28 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]': finished 2024-02-05T07:50:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: osdmap e726: 8 total, 8 up, 8 in 2024-02-05T07:50:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-144"}]: dispatch 2024-02-05T07:50:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:28 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:50:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:50:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:29 smithi043 ceph-mon[29879]: osdmap e727: 8 total, 8 up, 8 in 2024-02-05T07:50:29.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:29 smithi043 ceph-mon[29879]: pgmap v1131: 236 pgs: 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:29 smithi043 ceph-mon[41444]: osdmap e727: 8 total, 8 up, 8 in 2024-02-05T07:50:29.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:29 smithi043 ceph-mon[41444]: pgmap v1131: 236 pgs: 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:29 smithi050 ceph-mon[33492]: osdmap e727: 8 total, 8 up, 8 in 2024-02-05T07:50:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:29 smithi050 ceph-mon[33492]: pgmap v1131: 236 pgs: 236 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:50:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:30 smithi043 ceph-mon[29879]: osdmap e728: 8 total, 8 up, 8 in 2024-02-05T07:50:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:30 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:30 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:30 smithi043 ceph-mon[41444]: osdmap e728: 8 total, 8 up, 8 in 2024-02-05T07:50:30.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:30 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:30.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:30 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:30 smithi050 ceph-mon[33492]: osdmap e728: 8 total, 8 up, 8 in 2024-02-05T07:50:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:30 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:30 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[29879]: osdmap e729: 8 total, 8 up, 8 in 2024-02-05T07:50:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:31.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[29879]: pgmap v1134: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:50:31.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[41444]: osdmap e729: 8 total, 8 up, 8 in 2024-02-05T07:50:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:31.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:31 smithi043 ceph-mon[41444]: pgmap v1134: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:50:31.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:50:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:50:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:31 smithi050 ceph-mon[33492]: osdmap e729: 8 total, 8 up, 8 in 2024-02-05T07:50:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:31 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:31.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:31 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:31.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:31 smithi050 ceph-mon[33492]: pgmap v1134: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: osdmap e730: 8 total, 8 up, 8 in 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: osdmap e730: 8 total, 8 up, 8 in 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:32 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: osdmap e730: 8 total, 8 up, 8 in 2024-02-05T07:50:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:50:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:32 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]': finished 2024-02-05T07:50:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: osdmap e731: 8 total, 8 up, 8 in 2024-02-05T07:50:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]: dispatch 2024-02-05T07:50:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]: dispatch 2024-02-05T07:50:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:50:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:50:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[29879]: pgmap v1137: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]': finished 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: osdmap e731: 8 total, 8 up, 8 in 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]: dispatch 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]: dispatch 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:33 smithi043 ceph-mon[41444]: pgmap v1137: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81993-111", "overlaypool": "test-rados-api-smithi043-81993-146"}]': finished 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: osdmap e731: 8 total, 8 up, 8 in 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]: dispatch 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]: dispatch 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:33 smithi050 ceph-mon[33492]: pgmap v1137: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]': finished 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[29879]: osdmap e732: 8 total, 8 up, 8 in 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]': finished 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[41444]: osdmap e732: 8 total, 8 up, 8 in 2024-02-05T07:50:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:34 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:34 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-05T07:50:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81993-146", "mode": "writeback"}]': finished 2024-02-05T07:50:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:34 smithi050 ceph-mon[33492]: osdmap e732: 8 total, 8 up, 8 in 2024-02-05T07:50:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:34 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]: dispatch 2024-02-05T07:50:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:50:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:35 smithi050 ceph-mon[33492]: osdmap e733: 8 total, 8 up, 8 in 2024-02-05T07:50:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:35 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:35.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:35 smithi050 ceph-mon[33492]: pgmap v1140: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[29879]: osdmap e733: 8 total, 8 up, 8 in 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[29879]: pgmap v1140: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_count","val": "2"}]': finished 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[41444]: osdmap e733: 8 total, 8 up, 8 in 2024-02-05T07:50:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]: dispatch 2024-02-05T07:50:35.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:35 smithi043 ceph-mon[41444]: pgmap v1140: 268 pgs: 268 active+clean; 4.4 MiB data, 814 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:50:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:36 smithi050 ceph-mon[33492]: osdmap e734: 8 total, 8 up, 8 in 2024-02-05T07:50:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:36.573 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:50:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=sqlstore.transactions t=2024-02-05T07:50:36.280550095Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-05T07:50:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:50:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[41444]: osdmap e734: 8 total, 8 up, 8 in 2024-02-05T07:50:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:36.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:36.714 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_period","val": "600"}]': finished 2024-02-05T07:50:36.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[29879]: osdmap e734: 8 total, 8 up, 8 in 2024-02-05T07:50:36.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:36.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-05T07:50:37.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:37.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:50:37.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[29879]: osdmap e735: 8 total, 8 up, 8 in 2024-02-05T07:50:37.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:37.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:37.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[29879]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:37.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:37.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:50:37.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[41444]: osdmap e735: 8 total, 8 up, 8 in 2024-02-05T07:50:37.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:37.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:37.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:37 smithi043 ceph-mon[41444]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:37 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-05T07:50:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "hit_set_type","val": "bloom"}]': finished 2024-02-05T07:50:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:37 smithi050 ceph-mon[33492]: osdmap e735: 8 total, 8 up, 8 in 2024-02-05T07:50:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-05T07:50:37.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:37 smithi050 ceph-mon[33492]: pgmap v1143: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[29879]: osdmap e736: 8 total, 8 up, 8 in 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[41444]: osdmap e736: 8 total, 8 up, 8 in 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:50:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:50:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-05T07:50:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:38 smithi050 ceph-mon[33492]: osdmap e736: 8 total, 8 up, 8 in 2024-02-05T07:50:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:50:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]: dispatch 2024-02-05T07:50:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:50:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:39 smithi043 ceph-mon[29879]: osdmap e737: 8 total, 8 up, 8 in 2024-02-05T07:50:39.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:39 smithi043 ceph-mon[29879]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:50:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:50:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:50:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:39 smithi043 ceph-mon[41444]: osdmap e737: 8 total, 8 up, 8 in 2024-02-05T07:50:39.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:39 smithi043 ceph-mon[41444]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:39.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-146","var": "target_max_objects","val": "1"}]': finished 2024-02-05T07:50:39.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:39 smithi050 ceph-mon[33492]: osdmap e737: 8 total, 8 up, 8 in 2024-02-05T07:50:39.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:39 smithi050 ceph-mon[33492]: pgmap v1146: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:40 smithi050 ceph-mon[33492]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:50:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:40 smithi043 ceph-mon[29879]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:50:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:40 smithi043 ceph-mon[41444]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-05T07:50:41.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:50:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:50:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:42 smithi043 ceph-mon[29879]: pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:42 smithi043 ceph-mon[41444]: pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:42 smithi050 ceph-mon[33492]: pgmap v1147: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-05T07:50:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:44 smithi043 ceph-mon[29879]: pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 773 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:44 smithi043 ceph-mon[41444]: pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 773 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:44 smithi050 ceph-mon[33492]: pgmap v1148: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 773 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:46 smithi043 ceph-mon[29879]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:50:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:46 smithi043 ceph-mon[41444]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:50:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:46 smithi050 ceph-mon[33492]: pgmap v1149: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-02-05T07:50:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:48 smithi043 ceph-mon[29879]: pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:48 smithi043 ceph-mon[41444]: pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:48 smithi050 ceph-mon[33492]: pgmap v1150: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:49 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:49 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:49 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:49 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:50:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:50:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:49 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:49.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:49 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[29879]: pgmap v1151: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 965 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[29879]: osdmap e738: 8 total, 8 up, 8 in 2024-02-05T07:50:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[41444]: pgmap v1151: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 965 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[41444]: osdmap e738: 8 total, 8 up, 8 in 2024-02-05T07:50:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:50.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:50 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:50 smithi050 ceph-mon[33492]: pgmap v1151: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 965 B/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:50:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:50 smithi050 ceph-mon[33492]: osdmap e738: 8 total, 8 up, 8 in 2024-02-05T07:50:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:50 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:50.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:50 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]': finished 2024-02-05T07:50:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:51 smithi050 ceph-mon[33492]: osdmap e739: 8 total, 8 up, 8 in 2024-02-05T07:50:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:51.459 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:51 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:51.460 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:51 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]': finished 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[29879]: osdmap e739: 8 total, 8 up, 8 in 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]': finished 2024-02-05T07:50:51.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[41444]: osdmap e739: 8 total, 8 up, 8 in 2024-02-05T07:50:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:51.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:51 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-146"}]: dispatch 2024-02-05T07:50:51.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:50:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[41444]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[41444]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[41444]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[41444]: osdmap e740: 8 total, 8 up, 8 in 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[29879]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[29879]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[29879]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:50:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:52 smithi043 ceph-mon[29879]: osdmap e740: 8 total, 8 up, 8 in 2024-02-05T07:50:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:52 smithi050 ceph-mon[33492]: pgmap v1154: 268 pgs: 268 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-05T07:50:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:52 smithi050 ceph-mon[33492]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:50:52.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:52 smithi050 ceph-mon[33492]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-05T07:50:52.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:52 smithi050 ceph-mon[33492]: osdmap e740: 8 total, 8 up, 8 in 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[29879]: osdmap e741: 8 total, 8 up, 8 in 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[41444]: osdmap e741: 8 total, 8 up, 8 in 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:53 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:50:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:53 smithi050 ceph-mon[33492]: osdmap e741: 8 total, 8 up, 8 in 2024-02-05T07:50:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:53 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:53 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[29879]: pgmap v1157: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[29879]: osdmap e742: 8 total, 8 up, 8 in 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[41444]: pgmap v1157: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[41444]: osdmap e742: 8 total, 8 up, 8 in 2024-02-05T07:50:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:54.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:54 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:54 smithi050 ceph-mon[33492]: pgmap v1157: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-05T07:50:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:54 smithi050 ceph-mon[33492]: osdmap e742: 8 total, 8 up, 8 in 2024-02-05T07:50:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:54 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:54 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[29879]: osdmap e743: 8 total, 8 up, 8 in 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[41444]: osdmap e743: 8 total, 8 up, 8 in 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:55.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:55 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148", "force_nonempty": "--force-nonempty" }]': finished 2024-02-05T07:50:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:55 smithi050 ceph-mon[33492]: osdmap e743: 8 total, 8 up, 8 in 2024-02-05T07:50:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:55 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:55.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:55 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[41444]: pgmap v1160: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]': finished 2024-02-05T07:50:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[41444]: osdmap e744: 8 total, 8 up, 8 in 2024-02-05T07:50:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[29879]: pgmap v1160: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]': finished 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[29879]: osdmap e744: 8 total, 8 up, 8 in 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:56.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:56 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:56 smithi050 ceph-mon[33492]: pgmap v1160: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 815 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:50:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]': finished 2024-02-05T07:50:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:56 smithi050 ceph-mon[33492]: osdmap e744: 8 total, 8 up, 8 in 2024-02-05T07:50:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:56 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:56 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-148"}]: dispatch 2024-02-05T07:50:57.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:57 smithi043 ceph-mon[41444]: osdmap e745: 8 total, 8 up, 8 in 2024-02-05T07:50:57.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:57 smithi043 ceph-mon[29879]: osdmap e745: 8 total, 8 up, 8 in 2024-02-05T07:50:57.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:57 smithi050 ceph-mon[33492]: osdmap e745: 8 total, 8 up, 8 in 2024-02-05T07:50:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[29879]: pgmap v1163: 236 pgs: 236 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:50:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[29879]: osdmap e746: 8 total, 8 up, 8 in 2024-02-05T07:50:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:58.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[41444]: pgmap v1163: 236 pgs: 236 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:50:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[41444]: osdmap e746: 8 total, 8 up, 8 in 2024-02-05T07:50:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:58.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:58 smithi050 ceph-mon[33492]: pgmap v1163: 236 pgs: 236 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:50:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:58 smithi050 ceph-mon[33492]: osdmap e746: 8 total, 8 up, 8 in 2024-02-05T07:50:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:58 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:50:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:50:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:50:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:50:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:50:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[41444]: osdmap e747: 8 total, 8 up, 8 in 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-150"}]: dispatch 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-150"}]: dispatch 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[29879]: osdmap e747: 8 total, 8 up, 8 in 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:59.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-150"}]: dispatch 2024-02-05T07:50:59.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:50:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-150"}]: dispatch 2024-02-05T07:50:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:50:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:59 smithi050 ceph-mon[33492]: osdmap e747: 8 total, 8 up, 8 in 2024-02-05T07:50:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:50:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-150"}]: dispatch 2024-02-05T07:50:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:50:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-150"}]: dispatch 2024-02-05T07:51:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:00 smithi043 ceph-mon[41444]: pgmap v1166: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:51:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:00 smithi043 ceph-mon[41444]: osdmap e748: 8 total, 8 up, 8 in 2024-02-05T07:51:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:00 smithi043 ceph-mon[29879]: pgmap v1166: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:51:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:00 smithi043 ceph-mon[29879]: osdmap e748: 8 total, 8 up, 8 in 2024-02-05T07:51:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:00 smithi050 ceph-mon[33492]: pgmap v1166: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 804 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-05T07:51:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:00 smithi050 ceph-mon[33492]: osdmap e748: 8 total, 8 up, 8 in 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[29879]: osdmap e749: 8 total, 8 up, 8 in 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[29879]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[41444]: osdmap e749: 8 total, 8 up, 8 in 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:01.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:01 smithi043 ceph-mon[41444]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:51:01.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:51:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:51:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:01 smithi050 ceph-mon[33492]: osdmap e749: 8 total, 8 up, 8 in 2024-02-05T07:51:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:01.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:01 smithi050 ceph-mon[33492]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:51:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[29879]: pgmap v1169: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:51:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:51:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[29879]: osdmap e750: 8 total, 8 up, 8 in 2024-02-05T07:51:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-152"}]: dispatch 2024-02-05T07:51:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-152"}]: dispatch 2024-02-05T07:51:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[41444]: pgmap v1169: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:51:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:51:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[41444]: osdmap e750: 8 total, 8 up, 8 in 2024-02-05T07:51:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-152"}]: dispatch 2024-02-05T07:51:02.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-152"}]: dispatch 2024-02-05T07:51:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:02 smithi050 ceph-mon[33492]: pgmap v1169: 268 pgs: 32 creating+peering, 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:51:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:51:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:02 smithi050 ceph-mon[33492]: osdmap e750: 8 total, 8 up, 8 in 2024-02-05T07:51:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-152"}]: dispatch 2024-02-05T07:51:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-152"}]: dispatch 2024-02-05T07:51:03.515 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.515 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.515 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.515 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.515 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.515 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.516 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.517 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.518 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.519 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.520 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.521 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.522 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:51:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:03 smithi043 ceph-mon[29879]: osdmap e751: 8 total, 8 up, 8 in 2024-02-05T07:51:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:03 smithi043 ceph-mon[41444]: osdmap e751: 8 total, 8 up, 8 in 2024-02-05T07:51:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:03 smithi050 ceph-mon[33492]: osdmap e751: 8 total, 8 up, 8 in 2024-02-05T07:51:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[29879]: pgmap v1172: 236 pgs: 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:51:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[29879]: osdmap e752: 8 total, 8 up, 8 in 2024-02-05T07:51:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[41444]: pgmap v1172: 236 pgs: 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:51:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[41444]: osdmap e752: 8 total, 8 up, 8 in 2024-02-05T07:51:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:04.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:04 smithi050 ceph-mon[33492]: pgmap v1172: 236 pgs: 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-05T07:51:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:04 smithi050 ceph-mon[33492]: osdmap e752: 8 total, 8 up, 8 in 2024-02-05T07:51:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T07:51:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:51:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: osdmap e753: 8 total, 8 up, 8 in 2024-02-05T07:51:05.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-111","var": "dedup_tier","val": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-111","var": "dedup_tier","val": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[29879]: pgmap v1175: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:51:05.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: osdmap e753: 8 total, 8 up, 8 in 2024-02-05T07:51:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-111","var": "dedup_tier","val": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-111","var": "dedup_tier","val": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:05 smithi043 ceph-mon[41444]: pgmap v1175: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81993-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T07:51:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: osdmap e753: 8 total, 8 up, 8 in 2024-02-05T07:51:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-111","var": "dedup_tier","val": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81993-111","var": "dedup_tier","val": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81993-111", "tierpool": "test-rados-api-smithi043-81993-154"}]: dispatch 2024-02-05T07:51:05.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:05 smithi050 ceph-mon[33492]: pgmap v1175: 268 pgs: 32 unknown, 236 active+clean; 4.4 MiB data, 805 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:06 smithi043 ceph-mon[41444]: osdmap e754: 8 total, 8 up, 8 in 2024-02-05T07:51:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:06 smithi043 ceph-mon[29879]: osdmap e754: 8 total, 8 up, 8 in 2024-02-05T07:51:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:06 smithi050 ceph-mon[33492]: osdmap e754: 8 total, 8 up, 8 in 2024-02-05T07:51:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[29879]: osdmap e755: 8 total, 8 up, 8 in 2024-02-05T07:51:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:07.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[29879]: pgmap v1178: 228 pgs: 228 active+clean; 582 KiB data, 823 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[41444]: osdmap e755: 8 total, 8 up, 8 in 2024-02-05T07:51:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:07.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:07.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:07 smithi043 ceph-mon[41444]: pgmap v1178: 228 pgs: 228 active+clean; 582 KiB data, 823 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:07 smithi050 ceph-mon[33492]: osdmap e755: 8 total, 8 up, 8 in 2024-02-05T07:51:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:07 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:07.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:07 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:07.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:07 smithi050 ceph-mon[33492]: pgmap v1178: 228 pgs: 228 active+clean; 582 KiB data, 823 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:08.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waitLibRadosTwoPoolsECPP.TryFlush (7826 ms) 2024-02-05T07:51:08.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-05T07:51:08.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13065 ms) 2024-02-05T07:51:08.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8040 ms) 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13074 ms) 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7810 ms) 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8040 ms) 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 336:602f83fe:::foo:head 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9069 ms) 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,0 2024-02-05T07:51:08.314 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1707119383 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,1707119388,1707119389,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,1707119388,1707119389,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,1707119388,1707119389,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,1707119388,1707119389,1707119391,1707119392,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,1707119388,1707119389,1707119391,1707119392,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119383,1707119385,1707119386,1707119388,1707119389,1707119391,1707119392,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1707119386,1707119388,1707119389,1707119391,1707119392,1707119394,1707119395,0 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1707119386, trimmed 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20916 ms) 2024-02-05T07:51:08.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14037 ms) 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18077 ms) 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22910 ms) 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5031 ms) 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3025 ms) 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3011 ms) 2024-02-05T07:51:08.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-05T07:51:08.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3035 ms) 2024-02-05T07:51:08.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (232822 ms total) 2024-02-05T07:51:08.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-02-05T07:51:08.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-05T07:51:08.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (864527 ms total) 2024-02-05T07:51:08.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81670 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81670 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81848 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81848 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82457 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82457 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:51:08.318 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82127 2024-02-05T07:51:08.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82127 2024-02-05T07:51:08.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:51:08.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81990 2024-02-05T07:51:08.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81990 2024-02-05T07:51:08.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T07:51:08.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81753 2024-02-05T07:51:08.319 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81753 2024-02-05T07:51:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[29879]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:51:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:51:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[29879]: osdmap e756: 8 total, 8 up, 8 in 2024-02-05T07:51:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[41444]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:51:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:51:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[41444]: osdmap e756: 8 total, 8 up, 8 in 2024-02-05T07:51:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:08 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:08 smithi050 ceph-mon[33492]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T07:51:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:51:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:08 smithi050 ceph-mon[33492]: osdmap e756: 8 total, 8 up, 8 in 2024-02-05T07:51:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:08 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1335696141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:08 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]: dispatch 2024-02-05T07:51:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:09 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:51:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:09 smithi043 ceph-mon[29879]: osdmap e757: 8 total, 8 up, 8 in 2024-02-05T07:51:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:09 smithi043 ceph-mon[29879]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 823 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:51:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:51:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:09 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:51:09.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:09 smithi043 ceph-mon[41444]: osdmap e757: 8 total, 8 up, 8 in 2024-02-05T07:51:09.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:09 smithi043 ceph-mon[41444]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 823 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:09 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81993-111"}]': finished 2024-02-05T07:51:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:09 smithi050 ceph-mon[33492]: osdmap e757: 8 total, 8 up, 8 in 2024-02-05T07:51:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:09 smithi050 ceph-mon[33492]: pgmap v1181: 228 pgs: 228 active+clean; 582 KiB data, 823 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:11.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:51:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:51:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:12 smithi043 ceph-mon[29879]: pgmap v1182: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:12 smithi043 ceph-mon[41444]: pgmap v1182: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:12 smithi050 ceph-mon[33492]: pgmap v1182: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail 2024-02-05T07:51:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:13.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:14 smithi043 ceph-mon[29879]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 768 B/s rd, 0 op/s 2024-02-05T07:51:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:14 smithi043 ceph-mon[41444]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 768 B/s rd, 0 op/s 2024-02-05T07:51:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:14 smithi050 ceph-mon[33492]: pgmap v1183: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 768 B/s rd, 0 op/s 2024-02-05T07:51:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:16 smithi043 ceph-mon[29879]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:51:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:16 smithi043 ceph-mon[41444]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:51:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:16 smithi050 ceph-mon[33492]: pgmap v1184: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T07:51:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:18 smithi043 ceph-mon[29879]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T07:51:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:18 smithi043 ceph-mon[41444]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T07:51:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:18 smithi050 ceph-mon[33492]: pgmap v1185: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T07:51:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:51:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:51:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:20 smithi043 ceph-mon[29879]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 961 B/s rd, 0 op/s 2024-02-05T07:51:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:20 smithi043 ceph-mon[41444]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 961 B/s rd, 0 op/s 2024-02-05T07:51:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:20 smithi050 ceph-mon[33492]: pgmap v1186: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 961 B/s rd, 0 op/s 2024-02-05T07:51:21.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:51:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:51:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:22 smithi043 ceph-mon[29879]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:22 smithi043 ceph-mon[41444]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:22 smithi050 ceph-mon[33492]: pgmap v1187: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:24 smithi043 ceph-mon[29879]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:24 smithi043 ceph-mon[41444]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:24 smithi050 ceph-mon[33492]: pgmap v1188: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:26 smithi043 ceph-mon[41444]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:26.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:26 smithi043 ceph-mon[29879]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:26 smithi050 ceph-mon[33492]: pgmap v1189: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:28 smithi043 ceph-mon[41444]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:28.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:28 smithi043 ceph-mon[29879]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:28 smithi050 ceph-mon[33492]: pgmap v1190: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:51:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:51:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:29.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:30 smithi043 ceph-mon[41444]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:30 smithi043 ceph-mon[29879]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:30 smithi050 ceph-mon[33492]: pgmap v1191: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:31.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:51:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:51:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:32 smithi043 ceph-mon[41444]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:32 smithi043 ceph-mon[29879]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:32 smithi050 ceph-mon[33492]: pgmap v1192: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:51:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:33.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:33 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:51:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:33 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:51:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:33 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:51:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:33 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:51:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:33 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:51:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:34 smithi043 ceph-mon[29879]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:34 smithi043 ceph-mon[41444]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:34 smithi050 ceph-mon[33492]: pgmap v1193: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:36 smithi043 ceph-mon[29879]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:36 smithi043 ceph-mon[41444]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:36 smithi050 ceph-mon[33492]: pgmap v1194: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:38 smithi043 ceph-mon[41444]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:38 smithi043 ceph-mon[29879]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:38 smithi050 ceph-mon[33492]: pgmap v1195: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:51:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:51:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:40 smithi043 ceph-mon[29879]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:40 smithi043 ceph-mon[41444]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:40 smithi050 ceph-mon[33492]: pgmap v1196: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:41.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:51:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:51:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:42 smithi043 ceph-mon[29879]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:42 smithi043 ceph-mon[41444]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:42 smithi050 ceph-mon[33492]: pgmap v1197: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:44 smithi043 ceph-mon[29879]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:44 smithi043 ceph-mon[41444]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:44 smithi050 ceph-mon[33492]: pgmap v1198: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:46 smithi043 ceph-mon[29879]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:46 smithi043 ceph-mon[41444]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:46 smithi050 ceph-mon[33492]: pgmap v1199: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:48 smithi043 ceph-mon[41444]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:48 smithi043 ceph-mon[29879]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:48 smithi050 ceph-mon[33492]: pgmap v1200: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:51:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:51:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:51:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:50 smithi043 ceph-mon[29879]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:50 smithi043 ceph-mon[41444]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:50 smithi050 ceph-mon[33492]: pgmap v1201: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:51.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:51:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:51:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:52 smithi043 ceph-mon[29879]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:52 smithi043 ceph-mon[41444]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:52 smithi050 ceph-mon[33492]: pgmap v1202: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:51:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:54 smithi043 ceph-mon[29879]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:54 smithi043 ceph-mon[41444]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:54 smithi050 ceph-mon[33492]: pgmap v1203: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:51:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:56 smithi043 ceph-mon[29879]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:56 smithi043 ceph-mon[41444]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:56 smithi050 ceph-mon[33492]: pgmap v1204: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:51:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:58 smithi043 ceph-mon[29879]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:51:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:58 smithi043 ceph-mon[41444]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:51:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:58 smithi050 ceph-mon[33492]: pgmap v1205: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:51:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:51:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:51:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:51:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:51:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:00 smithi043 ceph-mon[41444]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:00 smithi043 ceph-mon[29879]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:00 smithi050 ceph-mon[33492]: pgmap v1206: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:01.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:52:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:52:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:02 smithi043 ceph-mon[41444]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:02 smithi043 ceph-mon[29879]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:02 smithi050 ceph-mon[33492]: pgmap v1207: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:04 smithi043 ceph-mon[29879]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:04 smithi043 ceph-mon[41444]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:04 smithi050 ceph-mon[33492]: pgmap v1208: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:06 smithi043 ceph-mon[29879]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:06 smithi043 ceph-mon[41444]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:06 smithi050 ceph-mon[33492]: pgmap v1209: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:08 smithi043 ceph-mon[29879]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:08 smithi043 ceph-mon[41444]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:08 smithi050 ceph-mon[33492]: pgmap v1210: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:52:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:52:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:10 smithi043 ceph-mon[29879]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:10 smithi043 ceph-mon[41444]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:10 smithi050 ceph-mon[33492]: pgmap v1211: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:11.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:52:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:52:12.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:12 smithi043 ceph-mon[29879]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:12.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:12 smithi043 ceph-mon[41444]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:12 smithi050 ceph-mon[33492]: pgmap v1212: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:14 smithi043 ceph-mon[29879]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:14 smithi043 ceph-mon[41444]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:14 smithi050 ceph-mon[33492]: pgmap v1213: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:16 smithi043 ceph-mon[29879]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:16 smithi043 ceph-mon[41444]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:16 smithi050 ceph-mon[33492]: pgmap v1214: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:18 smithi043 ceph-mon[29879]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:18 smithi043 ceph-mon[41444]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:18 smithi050 ceph-mon[33492]: pgmap v1215: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:52:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:52:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:20 smithi043 ceph-mon[41444]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:20 smithi043 ceph-mon[29879]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:20 smithi050 ceph-mon[33492]: pgmap v1216: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:21.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:52:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:52:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:22 smithi043 ceph-mon[41444]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:22 smithi043 ceph-mon[29879]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:22 smithi050 ceph-mon[33492]: pgmap v1217: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:24 smithi043 ceph-mon[41444]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:24 smithi043 ceph-mon[29879]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:24 smithi050 ceph-mon[33492]: pgmap v1218: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:26 smithi043 ceph-mon[29879]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:26 smithi043 ceph-mon[41444]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:26 smithi050 ceph-mon[33492]: pgmap v1219: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:28 smithi043 ceph-mon[29879]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:28 smithi043 ceph-mon[41444]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:28 smithi050 ceph-mon[33492]: pgmap v1220: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:29.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:52:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:52:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:30 smithi043 ceph-mon[29879]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:30 smithi043 ceph-mon[41444]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:30 smithi050 ceph-mon[33492]: pgmap v1221: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:31.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:52:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:52:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:32 smithi043 ceph-mon[29879]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:32 smithi043 ceph-mon[41444]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:32 smithi050 ceph-mon[33492]: pgmap v1222: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[29879]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:52:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:52:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:52:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:52:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[41444]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:52:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:52:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:52:34.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:34 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:52:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:34 smithi050 ceph-mon[33492]: pgmap v1223: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:34 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:52:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:34 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:52:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:34 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:52:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:34 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:52:36.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:36 smithi043 ceph-mon[29879]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:36 smithi043 ceph-mon[41444]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:36 smithi050 ceph-mon[33492]: pgmap v1224: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:38 smithi043 ceph-mon[29879]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:38 smithi043 ceph-mon[41444]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:38 smithi050 ceph-mon[33492]: pgmap v1225: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:52:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:52:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:40 smithi043 ceph-mon[29879]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:40 smithi043 ceph-mon[41444]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:40 smithi050 ceph-mon[33492]: pgmap v1226: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:41.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:52:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:52:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:42 smithi043 ceph-mon[29879]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:42 smithi043 ceph-mon[41444]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:42 smithi050 ceph-mon[33492]: pgmap v1227: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:43.528 INFO:tasks.workunit.client.0.smithi043.stdout:ing 1 sec 2024-02-05T07:52:43.528 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.529 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.530 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.531 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.532 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.533 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.534 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.535 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:52:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:44.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:44 smithi043 ceph-mon[41444]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:44.567 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:44.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:44 smithi043 ceph-mon[29879]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:44.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:44 smithi050 ceph-mon[33492]: pgmap v1228: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:46 smithi043 ceph-mon[29879]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:46 smithi043 ceph-mon[41444]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:46 smithi050 ceph-mon[33492]: pgmap v1229: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:48 smithi043 ceph-mon[29879]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:48 smithi043 ceph-mon[41444]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:48 smithi050 ceph-mon[33492]: pgmap v1230: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:52:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:52:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:50 smithi043 ceph-mon[29879]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:50 smithi043 ceph-mon[41444]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:50 smithi050 ceph-mon[33492]: pgmap v1231: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:51.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:52:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:52:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:52 smithi043 ceph-mon[41444]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:52 smithi043 ceph-mon[29879]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:52 smithi050 ceph-mon[33492]: pgmap v1232: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:52:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:54 smithi043 ceph-mon[29879]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:54 smithi043 ceph-mon[41444]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:54 smithi050 ceph-mon[33492]: pgmap v1233: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:52:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:56 smithi043 ceph-mon[29879]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:56 smithi043 ceph-mon[41444]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:56 smithi050 ceph-mon[33492]: pgmap v1234: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:52:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:58 smithi043 ceph-mon[29879]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:58 smithi043 ceph-mon[41444]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:58 smithi050 ceph-mon[33492]: pgmap v1235: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:52:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:52:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:52:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:52:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:52:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:52:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:52:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:52:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:00 smithi043 ceph-mon[29879]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:00 smithi043 ceph-mon[41444]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:00 smithi050 ceph-mon[33492]: pgmap v1236: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:01.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:53:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:53:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:02 smithi043 ceph-mon[29879]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:02 smithi043 ceph-mon[41444]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:02 smithi050 ceph-mon[33492]: pgmap v1237: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:03.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:04 smithi043 ceph-mon[29879]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:04 smithi043 ceph-mon[41444]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:04 smithi050 ceph-mon[33492]: pgmap v1238: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:06.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:06 smithi043 ceph-mon[29879]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:06.565 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:06 smithi043 ceph-mon[41444]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:06 smithi050 ceph-mon[33492]: pgmap v1239: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:08 smithi043 ceph-mon[29879]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:08 smithi043 ceph-mon[41444]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:08.578 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:08 smithi050 ceph-mon[33492]: pgmap v1240: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:09 smithi043 ceph-mon[29879]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:53:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:53:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:09.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:09 smithi043 ceph-mon[41444]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:09.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:09 smithi050 ceph-mon[33492]: pgmap v1241: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:11.823 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:53:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:53:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:12 smithi043 ceph-mon[29879]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:12 smithi043 ceph-mon[41444]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:12 smithi050 ceph-mon[33492]: pgmap v1242: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:14 smithi043 ceph-mon[29879]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:14 smithi043 ceph-mon[41444]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:14 smithi050 ceph-mon[33492]: pgmap v1243: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:16 smithi043 ceph-mon[29879]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:16 smithi043 ceph-mon[41444]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:16 smithi050 ceph-mon[33492]: pgmap v1244: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:18 smithi043 ceph-mon[29879]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:18 smithi043 ceph-mon[41444]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:18 smithi050 ceph-mon[33492]: pgmap v1245: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:53:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:53:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:20 smithi043 ceph-mon[29879]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:20 smithi043 ceph-mon[41444]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:20 smithi050 ceph-mon[33492]: pgmap v1246: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:21.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:53:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:53:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:22 smithi043 ceph-mon[29879]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:22 smithi043 ceph-mon[41444]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:22 smithi050 ceph-mon[33492]: pgmap v1247: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:24 smithi043 ceph-mon[29879]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:24 smithi043 ceph-mon[41444]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:24 smithi050 ceph-mon[33492]: pgmap v1248: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:26 smithi043 ceph-mon[29879]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:26.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:26 smithi043 ceph-mon[41444]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:26 smithi050 ceph-mon[33492]: pgmap v1249: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:28 smithi043 ceph-mon[29879]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:28 smithi043 ceph-mon[41444]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:28 smithi050 ceph-mon[33492]: pgmap v1250: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:53:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:53:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:30 smithi043 ceph-mon[29879]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:30 smithi043 ceph-mon[41444]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:30 smithi050 ceph-mon[33492]: pgmap v1251: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:32.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:53:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:53:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:32 smithi043 ceph-mon[29879]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:32 smithi043 ceph-mon[41444]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:32 smithi050 ceph-mon[33492]: pgmap v1252: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:34 smithi043 ceph-mon[29879]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:34 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:53:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:34 smithi043 ceph-mon[41444]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:34 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:53:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:34 smithi050 ceph-mon[33492]: pgmap v1253: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:34 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:53:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:35 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:53:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:35 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:53:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:35 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:53:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:35 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:53:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:35 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:53:35.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:35 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:53:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:35 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:53:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:35 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:53:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:35 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:53:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:36 smithi043 ceph-mon[29879]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:36 smithi043 ceph-mon[41444]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:36 smithi050 ceph-mon[33492]: pgmap v1254: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:38 smithi043 ceph-mon[29879]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:38 smithi043 ceph-mon[41444]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:38 smithi050 ceph-mon[33492]: pgmap v1255: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:53:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:53:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:40 smithi043 ceph-mon[29879]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:40 smithi043 ceph-mon[41444]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:40 smithi050 ceph-mon[33492]: pgmap v1256: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:42.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:53:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:53:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:42 smithi043 ceph-mon[29879]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:42 smithi043 ceph-mon[41444]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:42 smithi050 ceph-mon[33492]: pgmap v1257: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:44 smithi043 ceph-mon[29879]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:44 smithi043 ceph-mon[41444]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:44 smithi050 ceph-mon[33492]: pgmap v1258: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:46 smithi043 ceph-mon[29879]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:46 smithi043 ceph-mon[41444]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:46 smithi050 ceph-mon[33492]: pgmap v1259: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:48 smithi043 ceph-mon[29879]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:48 smithi043 ceph-mon[41444]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:48 smithi050 ceph-mon[33492]: pgmap v1260: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:53:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:53:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:50 smithi043 ceph-mon[29879]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:50 smithi043 ceph-mon[41444]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:50 smithi050 ceph-mon[33492]: pgmap v1261: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:53:52.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:53:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:53:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:52 smithi043 ceph-mon[29879]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:52 smithi043 ceph-mon[41444]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:52 smithi050 ceph-mon[33492]: pgmap v1262: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:53:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:54 smithi043 ceph-mon[29879]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:54 smithi043 ceph-mon[41444]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:54 smithi050 ceph-mon[33492]: pgmap v1263: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:56 smithi043 ceph-mon[29879]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:56 smithi043 ceph-mon[41444]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:56 smithi050 ceph-mon[33492]: pgmap v1264: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:53:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:58 smithi043 ceph-mon[29879]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:58 smithi043 ceph-mon[41444]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:58 smithi050 ceph-mon[33492]: pgmap v1265: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:53:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:53:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:53:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:53:59.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:53:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:53:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:53:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:53:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:00 smithi043 ceph-mon[41444]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:00 smithi043 ceph-mon[29879]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:00 smithi050 ceph-mon[33492]: pgmap v1266: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:02.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:54:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:54:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:02 smithi043 ceph-mon[41444]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:02 smithi043 ceph-mon[29879]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:02 smithi050 ceph-mon[33492]: pgmap v1267: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:04 smithi043 ceph-mon[41444]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:04 smithi043 ceph-mon[29879]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:04 smithi050 ceph-mon[33492]: pgmap v1268: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:06 smithi043 ceph-mon[29879]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:06 smithi043 ceph-mon[41444]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:06 smithi050 ceph-mon[33492]: pgmap v1269: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:08 smithi043 ceph-mon[29879]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 25 op/s 2024-02-05T07:54:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:08 smithi043 ceph-mon[41444]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 25 op/s 2024-02-05T07:54:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:08 smithi050 ceph-mon[33492]: pgmap v1270: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 25 op/s 2024-02-05T07:54:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:09 smithi043 ceph-mon[29879]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 24 op/s 2024-02-05T07:54:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:54:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:54:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:09 smithi043 ceph-mon[41444]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 24 op/s 2024-02-05T07:54:09.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:09 smithi050 ceph-mon[33492]: pgmap v1271: 228 pgs: 228 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 24 op/s 2024-02-05T07:54:12.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:54:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:54:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:12 smithi043 ceph-mon[29879]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:12 smithi043 ceph-mon[41444]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:12 smithi050 ceph-mon[33492]: pgmap v1272: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:14 smithi043 ceph-mon[41444]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:14 smithi043 ceph-mon[29879]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:14.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:14 smithi050 ceph-mon[33492]: pgmap v1273: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:16 smithi043 ceph-mon[41444]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:16 smithi043 ceph-mon[29879]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:16 smithi050 ceph-mon[33492]: pgmap v1274: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:18 smithi043 ceph-mon[41444]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:18 smithi043 ceph-mon[29879]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:18 smithi050 ceph-mon[33492]: pgmap v1275: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T07:54:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:54:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:54:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:20 smithi043 ceph-mon[29879]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 36 op/s 2024-02-05T07:54:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:20 smithi043 ceph-mon[41444]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 36 op/s 2024-02-05T07:54:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:20 smithi050 ceph-mon[33492]: pgmap v1276: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 36 op/s 2024-02-05T07:54:22.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:54:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:54:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:22 smithi043 ceph-mon[41444]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 36 op/s 2024-02-05T07:54:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:22 smithi043 ceph-mon[29879]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 36 op/s 2024-02-05T07:54:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:22 smithi050 ceph-mon[33492]: pgmap v1277: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 23 KiB/s rd, 0 B/s wr, 36 op/s 2024-02-05T07:54:23.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.543 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.544 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.545 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.546 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.547 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.548 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.548 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.548 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.548 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.548 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.548 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:54:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:24 smithi043 ceph-mon[29879]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:24 smithi043 ceph-mon[41444]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:24 smithi050 ceph-mon[33492]: pgmap v1278: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:26 smithi043 ceph-mon[29879]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:26 smithi043 ceph-mon[41444]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:26 smithi050 ceph-mon[33492]: pgmap v1279: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:28 smithi043 ceph-mon[29879]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:28 smithi043 ceph-mon[41444]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:28 smithi050 ceph-mon[33492]: pgmap v1280: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:54:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:54:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:30 smithi043 ceph-mon[29879]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:30 smithi043 ceph-mon[41444]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:30 smithi050 ceph-mon[33492]: pgmap v1281: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:32.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:54:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:54:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:32 smithi043 ceph-mon[29879]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:32.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:32 smithi043 ceph-mon[41444]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:32 smithi050 ceph-mon[33492]: pgmap v1282: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:34 smithi043 ceph-mon[41444]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:34 smithi043 ceph-mon[29879]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:34 smithi050 ceph-mon[33492]: pgmap v1283: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:35.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:35 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:54:35.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:35 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:54:35.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:35 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:54:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[29879]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:54:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:54:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:54:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[41444]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:54:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:54:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:36 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:54:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:36 smithi050 ceph-mon[33492]: pgmap v1284: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:36 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:54:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:36 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:54:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:36 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:54:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:38 smithi043 ceph-mon[29879]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:38 smithi043 ceph-mon[41444]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:38 smithi050 ceph-mon[33492]: pgmap v1285: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:54:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:54:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:40.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:40 smithi043 ceph-mon[29879]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:40.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:40 smithi043 ceph-mon[41444]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:40 smithi050 ceph-mon[33492]: pgmap v1286: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:42.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:54:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:54:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:42 smithi043 ceph-mon[29879]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:42 smithi043 ceph-mon[41444]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:42 smithi050 ceph-mon[33492]: pgmap v1287: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:44 smithi043 ceph-mon[29879]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:44 smithi043 ceph-mon[41444]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:44.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:44 smithi050 ceph-mon[33492]: pgmap v1288: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:46 smithi043 ceph-mon[29879]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:46 smithi043 ceph-mon[41444]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:46 smithi050 ceph-mon[33492]: pgmap v1289: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:48 smithi043 ceph-mon[29879]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:48 smithi043 ceph-mon[41444]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:48 smithi050 ceph-mon[33492]: pgmap v1290: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:54:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:54:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:50 smithi043 ceph-mon[29879]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:50 smithi043 ceph-mon[41444]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:50 smithi050 ceph-mon[33492]: pgmap v1291: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:52.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:54:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:54:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:52 smithi043 ceph-mon[29879]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:52 smithi043 ceph-mon[41444]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:52 smithi050 ceph-mon[33492]: pgmap v1292: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:54:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:54 smithi043 ceph-mon[29879]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:54 smithi043 ceph-mon[41444]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:54 smithi050 ceph-mon[33492]: pgmap v1293: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:54:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:56 smithi043 ceph-mon[29879]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:56 smithi043 ceph-mon[41444]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:56 smithi050 ceph-mon[33492]: pgmap v1294: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:54:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:58 smithi043 ceph-mon[29879]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:58 smithi043 ceph-mon[41444]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:58 smithi050 ceph-mon[33492]: pgmap v1295: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:54:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:54:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:54:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:54:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:54:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:54:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:54:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:54:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:00 smithi043 ceph-mon[41444]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:00 smithi043 ceph-mon[29879]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:00 smithi050 ceph-mon[33492]: pgmap v1296: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:02.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:55:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:55:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:02 smithi043 ceph-mon[29879]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:02 smithi043 ceph-mon[41444]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:02 smithi050 ceph-mon[33492]: pgmap v1297: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:03.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:03.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:03 smithi050 ceph-mon[33492]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:03 smithi043 ceph-mon[29879]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:03 smithi043 ceph-mon[41444]: pgmap v1298: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:06.516 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:06 smithi043 ceph-mon[29879]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:06.517 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:06 smithi043 ceph-mon[41444]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:06 smithi050 ceph-mon[33492]: pgmap v1299: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:08 smithi043 ceph-mon[29879]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:08 smithi043 ceph-mon[41444]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:08 smithi050 ceph-mon[33492]: pgmap v1300: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:55:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:55:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:10 smithi043 ceph-mon[41444]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:10 smithi043 ceph-mon[29879]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:10 smithi050 ceph-mon[33492]: pgmap v1301: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:12.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:55:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:55:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:12 smithi043 ceph-mon[41444]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:12 smithi043 ceph-mon[29879]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:12 smithi050 ceph-mon[33492]: pgmap v1302: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:14 smithi043 ceph-mon[41444]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:14 smithi043 ceph-mon[29879]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:14 smithi050 ceph-mon[33492]: pgmap v1303: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:16 smithi043 ceph-mon[29879]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:16.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:16 smithi043 ceph-mon[41444]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:16 smithi050 ceph-mon[33492]: pgmap v1304: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:18 smithi043 ceph-mon[29879]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:18 smithi043 ceph-mon[41444]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:18 smithi050 ceph-mon[33492]: pgmap v1305: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:55:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:55:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:20 smithi043 ceph-mon[29879]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:20 smithi043 ceph-mon[41444]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:20 smithi050 ceph-mon[33492]: pgmap v1306: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:22.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:55:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:55:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:22 smithi043 ceph-mon[29879]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:22 smithi043 ceph-mon[41444]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:22.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:22 smithi050 ceph-mon[33492]: pgmap v1307: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:24 smithi043 ceph-mon[29879]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:24 smithi043 ceph-mon[41444]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:24 smithi050 ceph-mon[33492]: pgmap v1308: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:26 smithi043 ceph-mon[41444]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:26 smithi043 ceph-mon[29879]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:26 smithi050 ceph-mon[33492]: pgmap v1309: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:28 smithi043 ceph-mon[29879]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:28 smithi043 ceph-mon[41444]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:28 smithi050 ceph-mon[33492]: pgmap v1310: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:29.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:55:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:55:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:30 smithi043 ceph-mon[29879]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:30 smithi043 ceph-mon[41444]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:30 smithi050 ceph-mon[33492]: pgmap v1311: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:55:32.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:55:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:55:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:32 smithi043 ceph-mon[29879]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:32 smithi043 ceph-mon[41444]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:32 smithi050 ceph-mon[33492]: pgmap v1312: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:34 smithi043 ceph-mon[29879]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:34 smithi043 ceph-mon[41444]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:34 smithi050 ceph-mon[33492]: pgmap v1313: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[29879]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:55:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:55:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:55:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:55:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[41444]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:55:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:55:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:55:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:36 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:55:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:36 smithi050 ceph-mon[33492]: pgmap v1314: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:36 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:55:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:36 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:55:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:36 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:55:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:36 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:55:37.323 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 07:55:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=cleanup t=2024-02-05T07:55:36.941938739Z level=info msg="Completed cleanup jobs" duration=2.68708ms 2024-02-05T07:55:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:38 smithi043 ceph-mon[29879]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:38.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:38 smithi043 ceph-mon[41444]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:38.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:38 smithi050 ceph-mon[33492]: pgmap v1315: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:55:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:55:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:40 smithi043 ceph-mon[41444]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:40 smithi043 ceph-mon[29879]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:40 smithi050 ceph-mon[33492]: pgmap v1316: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:42.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:55:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:55:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:42 smithi043 ceph-mon[29879]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:42.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:42 smithi043 ceph-mon[41444]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:42 smithi050 ceph-mon[33492]: pgmap v1317: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:43.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:44 smithi043 ceph-mon[41444]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:44 smithi043 ceph-mon[29879]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:44 smithi050 ceph-mon[33492]: pgmap v1318: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:46 smithi043 ceph-mon[29879]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:46 smithi043 ceph-mon[41444]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:46 smithi050 ceph-mon[33492]: pgmap v1319: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:48 smithi043 ceph-mon[29879]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:48.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:48 smithi043 ceph-mon[41444]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:48.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:48 smithi050 ceph-mon[33492]: pgmap v1320: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:55:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:55:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:50 smithi043 ceph-mon[29879]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:50 smithi043 ceph-mon[41444]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:50 smithi050 ceph-mon[33492]: pgmap v1321: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:52.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:55:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:55:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:52 smithi043 ceph-mon[29879]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:52 smithi043 ceph-mon[41444]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:52 smithi050 ceph-mon[33492]: pgmap v1322: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:53.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:53.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:55:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:54 smithi050 ceph-mon[33492]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:54.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:54 smithi043 ceph-mon[29879]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:54.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:54 smithi043 ceph-mon[41444]: pgmap v1323: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:55.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:55 smithi043 ceph-mon[29879]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:55.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:55 smithi043 ceph-mon[41444]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:55.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:55 smithi050 ceph-mon[33492]: pgmap v1324: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:55:58.526 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:58 smithi050 ceph-mon[33492]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:58 smithi043 ceph-mon[29879]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:58 smithi043 ceph-mon[41444]: pgmap v1325: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:55:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:55:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:55:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:55:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:55:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:55:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:55:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:55:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:00 smithi043 ceph-mon[29879]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:00 smithi043 ceph-mon[41444]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:00 smithi050 ceph-mon[33492]: pgmap v1326: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:02.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:56:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:56:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:02 smithi043 ceph-mon[29879]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:02 smithi043 ceph-mon[41444]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:02 smithi050 ceph-mon[33492]: pgmap v1327: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:03.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.556 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.557 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.558 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.559 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.560 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:56:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:04 smithi043 ceph-mon[29879]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:04 smithi043 ceph-mon[41444]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:04.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:04 smithi050 ceph-mon[33492]: pgmap v1328: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:06 smithi043 ceph-mon[29879]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:06 smithi043 ceph-mon[41444]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:06 smithi050 ceph-mon[33492]: pgmap v1329: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:08 smithi043 ceph-mon[29879]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:08 smithi043 ceph-mon[41444]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:08 smithi050 ceph-mon[33492]: pgmap v1330: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:56:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:56:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:10 smithi043 ceph-mon[29879]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:10 smithi043 ceph-mon[41444]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:10 smithi050 ceph-mon[33492]: pgmap v1331: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:12.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:56:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:56:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:12 smithi043 ceph-mon[29879]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:12 smithi043 ceph-mon[41444]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:12 smithi050 ceph-mon[33492]: pgmap v1332: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:13.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:14 smithi043 ceph-mon[29879]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:14 smithi043 ceph-mon[41444]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:14 smithi050 ceph-mon[33492]: pgmap v1333: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:14.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:16 smithi043 ceph-mon[29879]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:16 smithi043 ceph-mon[41444]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:16 smithi050 ceph-mon[33492]: pgmap v1334: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:18 smithi043 ceph-mon[29879]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:18 smithi043 ceph-mon[41444]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:18 smithi050 ceph-mon[33492]: pgmap v1335: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:56:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:56:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:20 smithi043 ceph-mon[29879]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:20 smithi043 ceph-mon[41444]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:20 smithi050 ceph-mon[33492]: pgmap v1336: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:22.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:56:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:56:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:22 smithi043 ceph-mon[29879]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:22 smithi043 ceph-mon[41444]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:22 smithi050 ceph-mon[33492]: pgmap v1337: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:24 smithi043 ceph-mon[29879]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:24 smithi043 ceph-mon[41444]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:24 smithi050 ceph-mon[33492]: pgmap v1338: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:26 smithi043 ceph-mon[29879]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:26 smithi043 ceph-mon[41444]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:26.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:26 smithi050 ceph-mon[33492]: pgmap v1339: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:28 smithi043 ceph-mon[41444]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:28 smithi043 ceph-mon[29879]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:28 smithi050 ceph-mon[33492]: pgmap v1340: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:29.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:56:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:56:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:29.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:30 smithi043 ceph-mon[29879]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:30 smithi043 ceph-mon[41444]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:30 smithi050 ceph-mon[33492]: pgmap v1341: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:32.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:56:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:56:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:32 smithi043 ceph-mon[41444]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:32 smithi043 ceph-mon[29879]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:32 smithi050 ceph-mon[33492]: pgmap v1342: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:34 smithi043 ceph-mon[29879]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:34 smithi043 ceph-mon[41444]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:34.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:34 smithi050 ceph-mon[33492]: pgmap v1343: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:56:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:36 smithi043 ceph-mon[29879]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:36 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:56:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:36 smithi043 ceph-mon[41444]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:36.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:36 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:56:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:36 smithi050 ceph-mon[33492]: pgmap v1344: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:36 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:56:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:37 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:56:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:37 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:56:37.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:37 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:56:37.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:37 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:56:37.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:37 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:56:37.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:37 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:56:37.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:37 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:56:37.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:37 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:56:37.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:37 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:56:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:38 smithi043 ceph-mon[29879]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:38 smithi043 ceph-mon[41444]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:38.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:38 smithi050 ceph-mon[33492]: pgmap v1345: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:56:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:56:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:40 smithi043 ceph-mon[29879]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:40 smithi043 ceph-mon[41444]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:40 smithi050 ceph-mon[33492]: pgmap v1346: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:42.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:56:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:56:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:42 smithi050 ceph-mon[33492]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:42 smithi043 ceph-mon[41444]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:42 smithi043 ceph-mon[29879]: pgmap v1347: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:44 smithi050 ceph-mon[33492]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:44 smithi043 ceph-mon[29879]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:44 smithi043 ceph-mon[41444]: pgmap v1348: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:46 smithi043 ceph-mon[41444]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:46.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:46 smithi043 ceph-mon[29879]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:46 smithi050 ceph-mon[33492]: pgmap v1349: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:47 smithi043 ceph-mon[41444]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:47 smithi043 ceph-mon[29879]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:47.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:47 smithi050 ceph-mon[33492]: pgmap v1350: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:56:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:56:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:50 smithi043 ceph-mon[41444]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:50 smithi043 ceph-mon[29879]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:50 smithi050 ceph-mon[33492]: pgmap v1351: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:52.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:56:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:56:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:52 smithi043 ceph-mon[29879]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:52 smithi043 ceph-mon[41444]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:52 smithi050 ceph-mon[33492]: pgmap v1352: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:53.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:56:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:54 smithi043 ceph-mon[41444]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:54 smithi043 ceph-mon[29879]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:54 smithi050 ceph-mon[33492]: pgmap v1353: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:56 smithi043 ceph-mon[29879]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:56 smithi043 ceph-mon[41444]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:56 smithi050 ceph-mon[33492]: pgmap v1354: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:56:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:58 smithi043 ceph-mon[29879]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:58 smithi043 ceph-mon[41444]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:58 smithi050 ceph-mon[33492]: pgmap v1355: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:56:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:56:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:56:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:56:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:56:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:56:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:56:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:56:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:00 smithi043 ceph-mon[29879]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:00 smithi043 ceph-mon[41444]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:00 smithi050 ceph-mon[33492]: pgmap v1356: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:02.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:57:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:57:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:02 smithi043 ceph-mon[29879]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:02 smithi043 ceph-mon[41444]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:02 smithi050 ceph-mon[33492]: pgmap v1357: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:03.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:04 smithi043 ceph-mon[29879]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:04 smithi043 ceph-mon[41444]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:04 smithi050 ceph-mon[33492]: pgmap v1358: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:06 smithi043 ceph-mon[29879]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:06.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:06 smithi043 ceph-mon[41444]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:06.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:06 smithi050 ceph-mon[33492]: pgmap v1359: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:08 smithi043 ceph-mon[29879]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:08 smithi043 ceph-mon[41444]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:08 smithi050 ceph-mon[33492]: pgmap v1360: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:57:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:57:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:10 smithi043 ceph-mon[29879]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:10 smithi043 ceph-mon[41444]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:10.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:10 smithi050 ceph-mon[33492]: pgmap v1361: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:12.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:57:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:57:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:12 smithi043 ceph-mon[29879]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:12.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:12 smithi043 ceph-mon[41444]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:12 smithi050 ceph-mon[33492]: pgmap v1362: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:14 smithi043 ceph-mon[29879]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:14 smithi043 ceph-mon[41444]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:14 smithi050 ceph-mon[33492]: pgmap v1363: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:16 smithi043 ceph-mon[41444]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:16 smithi043 ceph-mon[29879]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:16.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:16 smithi050 ceph-mon[33492]: pgmap v1364: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:18 smithi043 ceph-mon[29879]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:18 smithi043 ceph-mon[41444]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:18 smithi050 ceph-mon[33492]: pgmap v1365: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:57:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:57:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:20 smithi043 ceph-mon[29879]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:20 smithi043 ceph-mon[41444]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:20 smithi050 ceph-mon[33492]: pgmap v1366: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:22.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:57:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:57:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:22 smithi043 ceph-mon[29879]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:22 smithi043 ceph-mon[41444]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:22 smithi050 ceph-mon[33492]: pgmap v1367: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:23.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:23.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:24.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:24 smithi043 ceph-mon[29879]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:24.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:24 smithi043 ceph-mon[41444]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:24 smithi050 ceph-mon[33492]: pgmap v1368: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:26.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:26 smithi043 ceph-mon[29879]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:26.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:26 smithi043 ceph-mon[41444]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:26.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:26 smithi050 ceph-mon[33492]: pgmap v1369: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:28 smithi043 ceph-mon[29879]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:28 smithi043 ceph-mon[41444]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:28.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:28 smithi050 ceph-mon[33492]: pgmap v1370: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:57:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:57:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:30.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:30 smithi050 ceph-mon[33492]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:30 smithi043 ceph-mon[29879]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:30.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:30 smithi043 ceph-mon[41444]: pgmap v1371: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:32.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:57:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:57:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:32 smithi050 ceph-mon[33492]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:32 smithi043 ceph-mon[29879]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:32 smithi043 ceph-mon[41444]: pgmap v1372: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:33.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:33.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:34 smithi043 ceph-mon[29879]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:34 smithi043 ceph-mon[41444]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:34 smithi050 ceph-mon[33492]: pgmap v1373: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:36.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:36 smithi043 ceph-mon[29879]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:36.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:36 smithi043 ceph-mon[41444]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:36.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:36 smithi050 ceph-mon[33492]: pgmap v1374: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:37.786 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:37 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:57:37.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:37 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:57:37.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:37 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:57:38.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:38 smithi050 ceph-mon[33492]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:38 smithi043 ceph-mon[29879]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:38 smithi043 ceph-mon[41444]: pgmap v1375: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:39.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:57:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:57:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:39.650 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:39 smithi050 ceph-mon[33492]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:39.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:39 smithi043 ceph-mon[29879]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:39 smithi043 ceph-mon[41444]: pgmap v1376: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:57:41.777 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:41.778 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:41 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:41 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:41.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:41.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:41 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:42.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:57:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:57:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:42 smithi043 ceph-mon[29879]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:42 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:42 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:42 smithi043 ceph-mon[41444]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:42 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:42.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:42 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:42.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:42 smithi050 ceph-mon[33492]: pgmap v1377: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:42.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:42 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:42.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:42 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:43.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:57:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:57:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:57:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:57:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:57:43.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:57:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:57:43.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:57:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:44 smithi043 ceph-mon[29879]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:44 smithi043 ceph-mon[41444]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:44 smithi050 ceph-mon[33492]: pgmap v1378: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:44.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:46 smithi043 ceph-mon[29879]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:46 smithi043 ceph-mon[41444]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:46 smithi050 ceph-mon[33492]: pgmap v1379: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:48.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:48 smithi043 ceph-mon[29879]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:48 smithi043 ceph-mon[41444]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:48.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:48 smithi050 ceph-mon[33492]: pgmap v1380: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:49.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:49 smithi043 ceph-mon[41444]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:49.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:49 smithi043 ceph-mon[29879]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:57:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:57:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:49.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:49 smithi050 ceph-mon[33492]: pgmap v1381: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:52.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:57:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:57:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:52 smithi043 ceph-mon[29879]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:52 smithi043 ceph-mon[41444]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:52 smithi050 ceph-mon[33492]: pgmap v1382: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:57:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:54 smithi043 ceph-mon[41444]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:54 smithi043 ceph-mon[29879]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:54 smithi050 ceph-mon[33492]: pgmap v1383: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:56 smithi043 ceph-mon[41444]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:56 smithi043 ceph-mon[29879]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:56.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:56 smithi050 ceph-mon[33492]: pgmap v1384: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:57:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:58 smithi043 ceph-mon[29879]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:58 smithi043 ceph-mon[41444]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:58 smithi050 ceph-mon[33492]: pgmap v1385: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:57:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:57:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:57:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:57:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:57:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:57:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:57:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:57:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:00 smithi043 ceph-mon[29879]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:00 smithi043 ceph-mon[41444]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:00.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:00 smithi050 ceph-mon[33492]: pgmap v1386: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:02.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:58:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:58:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:02 smithi043 ceph-mon[29879]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:02 smithi043 ceph-mon[41444]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:02 smithi050 ceph-mon[33492]: pgmap v1387: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:03.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:03.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:03.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:04.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:04 smithi043 ceph-mon[29879]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:04.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:04 smithi043 ceph-mon[41444]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:04 smithi050 ceph-mon[33492]: pgmap v1388: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:06.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:06 smithi043 ceph-mon[29879]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:06.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:06 smithi043 ceph-mon[41444]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:06 smithi050 ceph-mon[33492]: pgmap v1389: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:08 smithi043 ceph-mon[29879]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:08.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:08 smithi043 ceph-mon[41444]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:08.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:08 smithi050 ceph-mon[33492]: pgmap v1390: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:58:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:58:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:10 smithi043 ceph-mon[29879]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:10 smithi043 ceph-mon[41444]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:10 smithi050 ceph-mon[33492]: pgmap v1391: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:12.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:58:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:58:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:12 smithi043 ceph-mon[29879]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:12 smithi043 ceph-mon[41444]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:12.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:12 smithi050 ceph-mon[33492]: pgmap v1392: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:14 smithi043 ceph-mon[29879]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:14 smithi043 ceph-mon[41444]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:14.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:14 smithi050 ceph-mon[33492]: pgmap v1393: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:16 smithi043 ceph-mon[29879]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:16 smithi043 ceph-mon[41444]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:16 smithi050 ceph-mon[33492]: pgmap v1394: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:18 smithi043 ceph-mon[29879]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:18 smithi043 ceph-mon[41444]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:18 smithi050 ceph-mon[33492]: pgmap v1395: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:58:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:58:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:20.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:20 smithi050 ceph-mon[33492]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:20 smithi043 ceph-mon[29879]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:20 smithi043 ceph-mon[41444]: pgmap v1396: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:22.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:58:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:58:22.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:22 smithi050 ceph-mon[33492]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:22 smithi043 ceph-mon[41444]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:22 smithi043 ceph-mon[29879]: pgmap v1397: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:23.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:24.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:24 smithi050 ceph-mon[33492]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:24 smithi043 ceph-mon[41444]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:24 smithi043 ceph-mon[29879]: pgmap v1398: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:26 smithi043 ceph-mon[41444]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:26 smithi043 ceph-mon[29879]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:26 smithi050 ceph-mon[33492]: pgmap v1399: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:28.625 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:28 smithi050 ceph-mon[33492]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:28 smithi043 ceph-mon[29879]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:28 smithi043 ceph-mon[41444]: pgmap v1400: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:29.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:58:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:58:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:30 smithi043 ceph-mon[29879]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:30.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:30 smithi043 ceph-mon[41444]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:30.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:30 smithi050 ceph-mon[33492]: pgmap v1401: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:32.073 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:58:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:58:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:32 smithi043 ceph-mon[29879]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:32 smithi043 ceph-mon[41444]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:32.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:32 smithi050 ceph-mon[33492]: pgmap v1402: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:33.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:34 smithi043 ceph-mon[41444]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:34.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:34 smithi043 ceph-mon[29879]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:34.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:34 smithi050 ceph-mon[33492]: pgmap v1403: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:35.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:35 smithi043 ceph-mon[29879]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:35 smithi043 ceph-mon[41444]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:35.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:35 smithi050 ceph-mon[33492]: pgmap v1404: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-05T07:58:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:38 smithi043 ceph-mon[29879]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:38.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:38 smithi043 ceph-mon[41444]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:38 smithi050 ceph-mon[33492]: pgmap v1405: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:58:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:58:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:40 smithi043 ceph-mon[41444]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:40 smithi043 ceph-mon[29879]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:40 smithi050 ceph-mon[33492]: pgmap v1406: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:42.275 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:58:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:58:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:42 smithi043 ceph-mon[29879]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:42 smithi043 ceph-mon[41444]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:42 smithi050 ceph-mon[33492]: pgmap v1407: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:58:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:58:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:58:43.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:58:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:58:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:58:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:58:43.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:43 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:58:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:58:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:58:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:43 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:58:43.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:43 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:58:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:44 smithi043 ceph-mon[29879]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:44.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:44 smithi043 ceph-mon[41444]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:44.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:44.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:44 smithi050 ceph-mon[33492]: pgmap v1408: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:46.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:46 smithi043 ceph-mon[41444]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:46.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:46 smithi043 ceph-mon[29879]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:46.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:46 smithi050 ceph-mon[33492]: pgmap v1409: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:48.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:48 smithi043 ceph-mon[29879]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:48 smithi043 ceph-mon[41444]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:48.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:48 smithi050 ceph-mon[33492]: pgmap v1410: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:58:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:58:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:50.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:50 smithi043 ceph-mon[29879]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:50.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:50 smithi043 ceph-mon[41444]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:50.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:50 smithi050 ceph-mon[33492]: pgmap v1411: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:58:52.291 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:58:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:58:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:52 smithi043 ceph-mon[41444]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:52 smithi043 ceph-mon[29879]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:52 smithi050 ceph-mon[33492]: pgmap v1412: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:53.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:58:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:54 smithi043 ceph-mon[29879]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:54 smithi043 ceph-mon[41444]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:54.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:54 smithi050 ceph-mon[33492]: pgmap v1413: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:56 smithi043 ceph-mon[29879]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:56 smithi043 ceph-mon[41444]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:56 smithi050 ceph-mon[33492]: pgmap v1414: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:58:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:58 smithi043 ceph-mon[29879]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:58 smithi043 ceph-mon[41444]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:58.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:58 smithi050 ceph-mon[33492]: pgmap v1415: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:58:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:58:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:58:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:58:59.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:58:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:58:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:58:59.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:58:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:00.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:00 smithi043 ceph-mon[29879]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:00.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:00 smithi043 ceph-mon[41444]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:00 smithi050 ceph-mon[33492]: pgmap v1416: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:02.312 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:59:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:59:02.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:02 smithi043 ceph-mon[29879]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:02.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:02 smithi043 ceph-mon[41444]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:02 smithi050 ceph-mon[33492]: pgmap v1417: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:03.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:04.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:04 smithi050 ceph-mon[33492]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:04 smithi043 ceph-mon[29879]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:04 smithi043 ceph-mon[41444]: pgmap v1418: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:06.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:06 smithi050 ceph-mon[33492]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:06 smithi043 ceph-mon[29879]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:06 smithi043 ceph-mon[41444]: pgmap v1419: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:08.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:08 smithi043 ceph-mon[41444]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:08.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:08 smithi043 ceph-mon[29879]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:08.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:08 smithi050 ceph-mon[33492]: pgmap v1420: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:59:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:59:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:10 smithi043 ceph-mon[29879]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:10 smithi043 ceph-mon[41444]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:10.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:10 smithi050 ceph-mon[33492]: pgmap v1421: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:12.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:59:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:59:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:12 smithi043 ceph-mon[29879]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:12 smithi043 ceph-mon[41444]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:12 smithi050 ceph-mon[33492]: pgmap v1422: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:13.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:14 smithi043 ceph-mon[29879]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:14 smithi043 ceph-mon[41444]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:14.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:14 smithi050 ceph-mon[33492]: pgmap v1423: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:14.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:16 smithi043 ceph-mon[29879]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:16 smithi043 ceph-mon[41444]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:16.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:16 smithi050 ceph-mon[33492]: pgmap v1424: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:18 smithi043 ceph-mon[29879]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:18 smithi043 ceph-mon[41444]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:18.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:18 smithi050 ceph-mon[33492]: pgmap v1425: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:59:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:59:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:20 smithi043 ceph-mon[29879]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:20 smithi043 ceph-mon[41444]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:20 smithi050 ceph-mon[33492]: pgmap v1426: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:22.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:59:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:59:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:22 smithi043 ceph-mon[29879]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:22 smithi043 ceph-mon[41444]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:22 smithi050 ceph-mon[33492]: pgmap v1427: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:23.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.577 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.578 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.579 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.580 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.581 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T07:59:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:24 smithi043 ceph-mon[29879]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:24 smithi043 ceph-mon[41444]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:24.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:24 smithi050 ceph-mon[33492]: pgmap v1428: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:25.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:25 smithi043 ceph-mon[29879]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:25.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:25 smithi043 ceph-mon[41444]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:25.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:25 smithi050 ceph-mon[33492]: pgmap v1429: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:28.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:28 smithi043 ceph-mon[29879]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:28.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:28 smithi043 ceph-mon[41444]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:28.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:28 smithi050 ceph-mon[33492]: pgmap v1430: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:59:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:59:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:29.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:29.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:30 smithi043 ceph-mon[29879]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:30.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:30 smithi043 ceph-mon[41444]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:30.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:30 smithi050 ceph-mon[33492]: pgmap v1431: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:32.290 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:59:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:59:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:32 smithi043 ceph-mon[29879]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:32 smithi043 ceph-mon[41444]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:32.575 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:32 smithi050 ceph-mon[33492]: pgmap v1432: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:33.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:33.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:33.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:34.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:34 smithi043 ceph-mon[29879]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:34.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:34 smithi043 ceph-mon[41444]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:34 smithi050 ceph-mon[33492]: pgmap v1433: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:36.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:36 smithi043 ceph-mon[29879]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:36.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:36 smithi043 ceph-mon[41444]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:36.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:36 smithi050 ceph-mon[33492]: pgmap v1434: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:38.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:38 smithi043 ceph-mon[29879]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:38.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:38 smithi043 ceph-mon[41444]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:38 smithi050 ceph-mon[33492]: pgmap v1435: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:59:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:59:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:40.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:40 smithi043 ceph-mon[29879]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:40.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:40 smithi043 ceph-mon[41444]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:40.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:40 smithi050 ceph-mon[33492]: pgmap v1436: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:42.309 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:59:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:59:42.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:42 smithi043 ceph-mon[29879]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:42.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:42 smithi043 ceph-mon[41444]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:42.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:42 smithi050 ceph-mon[33492]: pgmap v1437: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:43.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:44 smithi050 ceph-mon[33492]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:59:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:44.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[29879]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:59:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[41444]: pgmap v1438: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T07:59:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T07:59:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:44.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:45.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:59:45.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:59:45.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:59:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:59:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T07:59:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T07:59:45.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T07:59:46.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:46 smithi050 ceph-mon[33492]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:46 smithi043 ceph-mon[29879]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:46 smithi043 ceph-mon[41444]: pgmap v1439: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:48.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:48 smithi050 ceph-mon[33492]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:48.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:48 smithi043 ceph-mon[29879]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:48 smithi043 ceph-mon[41444]: pgmap v1440: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:59:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:59:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:50.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:50 smithi043 ceph-mon[29879]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:50 smithi043 ceph-mon[41444]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:50.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:50 smithi050 ceph-mon[33492]: pgmap v1441: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:52.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 07:59:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T07:59:52.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:52 smithi043 ceph-mon[29879]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:52.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:52 smithi043 ceph-mon[41444]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:52.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:52 smithi050 ceph-mon[33492]: pgmap v1442: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:53.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:53.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:53.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T07:59:54.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:54 smithi043 ceph-mon[29879]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:54.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:54 smithi043 ceph-mon[41444]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:54.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:54 smithi050 ceph-mon[33492]: pgmap v1443: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:56 smithi043 ceph-mon[41444]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:56 smithi043 ceph-mon[29879]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:56 smithi050 ceph-mon[33492]: pgmap v1444: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T07:59:58.625 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:58 smithi050 ceph-mon[33492]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:58 smithi043 ceph-mon[29879]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:58 smithi043 ceph-mon[41444]: pgmap v1445: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T07:59:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 07:59:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 07:59:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T07:59:59.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 07:59:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:07:59:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T07:59:59.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 07:59:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[29879]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[29879]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[29879]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[29879]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[29879]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[29879]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[29879]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[41444]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[41444]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[41444]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[41444]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[41444]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[41444]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T08:00:00.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:00 smithi043 ceph-mon[41444]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:00:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:00 smithi050 ceph-mon[33492]: pgmap v1446: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:00.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:00 smithi050 ceph-mon[33492]: Health detail: HEALTH_WARN 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:00 smithi050 ceph-mon[33492]: [WRN] POOL_APP_NOT_ENABLED: 3 pool(s) do not have an application enabled 2024-02-05T08:00:00.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:00 smithi050 ceph-mon[33492]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:00:00.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:00 smithi050 ceph-mon[33492]: application not enabled on pool 'poolrace.1804289383' 2024-02-05T08:00:00.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:00 smithi050 ceph-mon[33492]: application not enabled on pool 'poolrace2.846930886' 2024-02-05T08:00:00.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:00 smithi050 ceph-mon[33492]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:00:02.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:00:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:00:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:02 smithi043 ceph-mon[41444]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:02 smithi043 ceph-mon[29879]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:02 smithi050 ceph-mon[33492]: pgmap v1447: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:03.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:04 smithi043 ceph-mon[29879]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:04 smithi043 ceph-mon[41444]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:04 smithi050 ceph-mon[33492]: pgmap v1448: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:05 smithi043 ceph-mon[29879]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:05 smithi043 ceph-mon[41444]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:05 smithi050 ceph-mon[33492]: pgmap v1449: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:08.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:08 smithi043 ceph-mon[29879]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:08.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:08 smithi043 ceph-mon[41444]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:08.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:08 smithi050 ceph-mon[33492]: pgmap v1450: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:00:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:00:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:10.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:10 smithi043 ceph-mon[41444]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:10.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:10 smithi043 ceph-mon[29879]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:10.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:10 smithi050 ceph-mon[33492]: pgmap v1451: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:12.295 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:00:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:00:12.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:12 smithi043 ceph-mon[29879]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:12.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:12 smithi043 ceph-mon[41444]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:12 smithi050 ceph-mon[33492]: pgmap v1452: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:13.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:13.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:13.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:14 smithi043 ceph-mon[29879]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:14.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:14 smithi043 ceph-mon[41444]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:14.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:14 smithi050 ceph-mon[33492]: pgmap v1453: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:14.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:16.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:16 smithi043 ceph-mon[29879]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:16.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:16 smithi043 ceph-mon[41444]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:16.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:16 smithi050 ceph-mon[33492]: pgmap v1454: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:18.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:18 smithi043 ceph-mon[29879]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:18.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:18 smithi043 ceph-mon[41444]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:18.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:18 smithi050 ceph-mon[33492]: pgmap v1455: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:00:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:00:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:20.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:20 smithi043 ceph-mon[29879]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:20.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:20 smithi043 ceph-mon[41444]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:20.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:20 smithi050 ceph-mon[33492]: pgmap v1456: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:22.312 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:00:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:00:22.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:22 smithi043 ceph-mon[29879]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:22.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:22 smithi043 ceph-mon[41444]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:22.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:22 smithi050 ceph-mon[33492]: pgmap v1457: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:23.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:24.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:24 smithi050 ceph-mon[33492]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:24 smithi043 ceph-mon[29879]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:24 smithi043 ceph-mon[41444]: pgmap v1458: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:26.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:26 smithi050 ceph-mon[33492]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:26 smithi043 ceph-mon[41444]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:26 smithi043 ceph-mon[29879]: pgmap v1459: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:28 smithi043 ceph-mon[29879]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:28 smithi043 ceph-mon[41444]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:28 smithi050 ceph-mon[33492]: pgmap v1460: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:29.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:00:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:00:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:30.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:30 smithi043 ceph-mon[29879]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:30.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:30 smithi043 ceph-mon[41444]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:30.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:30 smithi050 ceph-mon[33492]: pgmap v1461: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:32.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:00:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:00:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:32 smithi043 ceph-mon[29879]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:32 smithi043 ceph-mon[41444]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:32.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:32 smithi050 ceph-mon[33492]: pgmap v1462: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:33.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:34 smithi043 ceph-mon[29879]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:34 smithi043 ceph-mon[41444]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:34 smithi050 ceph-mon[33492]: pgmap v1463: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:36.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:36 smithi043 ceph-mon[29879]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:36 smithi043 ceph-mon[41444]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:36 smithi050 ceph-mon[33492]: pgmap v1464: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:38.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:38 smithi043 ceph-mon[29879]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:38.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:38 smithi043 ceph-mon[41444]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:38.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:38 smithi050 ceph-mon[33492]: pgmap v1465: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:00:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:00:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:40 smithi043 ceph-mon[29879]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:40 smithi043 ceph-mon[41444]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:40 smithi050 ceph-mon[33492]: pgmap v1466: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:42.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:00:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:00:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:42 smithi043 ceph-mon[29879]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:42 smithi043 ceph-mon[41444]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:42 smithi050 ceph-mon[33492]: pgmap v1467: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:44 smithi043 ceph-mon[29879]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:44 smithi043 ceph-mon[41444]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:44 smithi050 ceph-mon[33492]: pgmap v1468: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:00:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:00:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:00:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:45.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[29879]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:00:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:00:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[41444]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:00:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:00:46.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:46 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:46 smithi050 ceph-mon[33492]: pgmap v1469: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:46 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:00:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:46 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:00:46.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:46 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:00:48.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:48 smithi043 ceph-mon[29879]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:48 smithi043 ceph-mon[41444]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:48 smithi050 ceph-mon[33492]: pgmap v1470: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:49.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:00:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:00:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:50.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:50 smithi043 ceph-mon[29879]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:50 smithi043 ceph-mon[41444]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:50 smithi050 ceph-mon[33492]: pgmap v1471: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:52.299 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:00:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:00:52.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:52 smithi043 ceph-mon[41444]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:52.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:52 smithi043 ceph-mon[29879]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:52 smithi050 ceph-mon[33492]: pgmap v1472: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:53.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:53.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:53.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:00:54.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:54 smithi043 ceph-mon[29879]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:54.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:54 smithi043 ceph-mon[41444]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:54 smithi050 ceph-mon[33492]: pgmap v1473: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:00:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:56 smithi043 ceph-mon[29879]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:56 smithi043 ceph-mon[41444]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:56 smithi050 ceph-mon[33492]: pgmap v1474: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:00:58.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:58 smithi043 ceph-mon[29879]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:58.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:58 smithi043 ceph-mon[41444]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:58.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:58 smithi050 ceph-mon[33492]: pgmap v1475: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:00:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:00:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:00:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:00:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:00:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:00:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:00:59.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:00:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:00.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:00 smithi050 ceph-mon[33492]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:00 smithi043 ceph-mon[41444]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:00 smithi043 ceph-mon[29879]: pgmap v1476: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:02.320 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:01:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:01:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:02 smithi050 ceph-mon[33492]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:02 smithi043 ceph-mon[41444]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:02 smithi043 ceph-mon[29879]: pgmap v1477: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:03.589 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.589 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.589 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.589 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.590 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.591 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.592 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.593 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.594 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.595 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.596 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:01:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:04 smithi043 ceph-mon[29879]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:04 smithi043 ceph-mon[41444]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:04 smithi050 ceph-mon[33492]: pgmap v1478: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:06 smithi043 ceph-mon[29879]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:06 smithi043 ceph-mon[41444]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:06 smithi050 ceph-mon[33492]: pgmap v1479: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:08.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:08 smithi043 ceph-mon[41444]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:08.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:08 smithi043 ceph-mon[29879]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:08 smithi050 ceph-mon[33492]: pgmap v1480: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:01:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:01:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:10 smithi043 ceph-mon[41444]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:10.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:10 smithi043 ceph-mon[29879]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:10 smithi050 ceph-mon[33492]: pgmap v1481: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:12.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:01:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:01:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:12 smithi043 ceph-mon[41444]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:12 smithi043 ceph-mon[29879]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:12.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:12 smithi050 ceph-mon[33492]: pgmap v1482: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:13.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:13.815 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:14 smithi043 ceph-mon[29879]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:14 smithi043 ceph-mon[41444]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:14 smithi050 ceph-mon[33492]: pgmap v1483: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:16.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:16 smithi043 ceph-mon[29879]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:16 smithi043 ceph-mon[41444]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:16 smithi050 ceph-mon[33492]: pgmap v1484: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:18 smithi043 ceph-mon[29879]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:18 smithi043 ceph-mon[41444]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:18 smithi050 ceph-mon[33492]: pgmap v1485: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:01:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:01:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:20 smithi043 ceph-mon[29879]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:20 smithi043 ceph-mon[41444]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:20 smithi050 ceph-mon[33492]: pgmap v1486: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:22.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:01:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:01:22.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:22 smithi043 ceph-mon[29879]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:22 smithi043 ceph-mon[41444]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:22 smithi050 ceph-mon[33492]: pgmap v1487: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:24 smithi043 ceph-mon[29879]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:24 smithi043 ceph-mon[41444]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:24 smithi050 ceph-mon[33492]: pgmap v1488: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:26 smithi043 ceph-mon[29879]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:26 smithi043 ceph-mon[41444]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:26 smithi050 ceph-mon[33492]: pgmap v1489: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:28.625 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:28 smithi050 ceph-mon[33492]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:28 smithi043 ceph-mon[41444]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:28 smithi043 ceph-mon[29879]: pgmap v1490: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:01:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:01:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:29 smithi043 ceph-mon[29879]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:29 smithi043 ceph-mon[41444]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:29.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:29 smithi050 ceph-mon[33492]: pgmap v1491: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:32.308 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:01:31 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:01:32.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:32 smithi043 ceph-mon[29879]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:32.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:32 smithi043 ceph-mon[41444]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:32 smithi050 ceph-mon[33492]: pgmap v1492: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:33.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:34.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:34 smithi050 ceph-mon[33492]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:34.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:34 smithi043 ceph-mon[29879]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:34 smithi043 ceph-mon[41444]: pgmap v1493: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:36.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:36 smithi050 ceph-mon[33492]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:36 smithi043 ceph-mon[41444]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:36.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:36 smithi043 ceph-mon[29879]: pgmap v1494: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:38.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:38 smithi050 ceph-mon[33492]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:38.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:38 smithi043 ceph-mon[29879]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:38.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:38 smithi043 ceph-mon[41444]: pgmap v1495: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:39.564 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:01:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:01:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:40 smithi043 ceph-mon[29879]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:40 smithi043 ceph-mon[41444]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:40 smithi050 ceph-mon[33492]: pgmap v1496: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:42.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:01:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:01:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:42 smithi043 ceph-mon[29879]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:42 smithi043 ceph-mon[41444]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:42.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:42 smithi050 ceph-mon[33492]: pgmap v1497: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:44 smithi043 ceph-mon[29879]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:44 smithi043 ceph-mon[41444]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:44 smithi050 ceph-mon[33492]: pgmap v1498: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:46 smithi043 ceph-mon[29879]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:46 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:01:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:46 smithi043 ceph-mon[41444]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:46 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:01:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:46 smithi050 ceph-mon[33492]: pgmap v1499: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:46 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:01:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:47 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:01:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:47 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:01:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:47 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:01:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:47 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:01:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:47 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:01:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:47 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:01:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:47 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:01:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:47 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:01:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:47 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:01:48.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:48 smithi043 ceph-mon[29879]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:48 smithi043 ceph-mon[41444]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:48 smithi050 ceph-mon[33492]: pgmap v1500: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:01:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:01:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:50.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:50 smithi043 ceph-mon[29879]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:50 smithi043 ceph-mon[41444]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:50 smithi050 ceph-mon[33492]: pgmap v1501: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:01:52.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:01:51 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:01:52.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:52 smithi043 ceph-mon[29879]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:52.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:52 smithi043 ceph-mon[41444]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:52.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:52 smithi050 ceph-mon[33492]: pgmap v1502: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:53.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:53.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:53.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:01:54.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:54 smithi043 ceph-mon[41444]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:54.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:54 smithi043 ceph-mon[29879]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:54.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:54 smithi050 ceph-mon[33492]: pgmap v1503: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:56 smithi043 ceph-mon[29879]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:56 smithi043 ceph-mon[41444]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:56 smithi050 ceph-mon[33492]: pgmap v1504: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:01:58.624 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:58 smithi050 ceph-mon[33492]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:58 smithi043 ceph-mon[29879]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:58 smithi043 ceph-mon[41444]: pgmap v1505: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:01:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:01:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:01:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:01:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:01:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:01:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:01:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:01:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:00 smithi043 ceph-mon[29879]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:00 smithi043 ceph-mon[41444]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:00 smithi050 ceph-mon[33492]: pgmap v1506: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:02.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:02:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:02:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:02 smithi043 ceph-mon[29879]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:02 smithi043 ceph-mon[41444]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:02 smithi050 ceph-mon[33492]: pgmap v1507: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:04 smithi043 ceph-mon[41444]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:04 smithi043 ceph-mon[29879]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:04 smithi050 ceph-mon[33492]: pgmap v1508: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:06 smithi043 ceph-mon[41444]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:06 smithi043 ceph-mon[29879]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:06 smithi050 ceph-mon[33492]: pgmap v1509: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:08 smithi043 ceph-mon[29879]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:08 smithi043 ceph-mon[41444]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:08 smithi050 ceph-mon[33492]: pgmap v1510: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:09.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:09 smithi043 ceph-mon[29879]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:02:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:02:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:09.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:09 smithi043 ceph-mon[41444]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:09.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:09 smithi050 ceph-mon[33492]: pgmap v1511: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:12.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:02:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:02:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:12 smithi050 ceph-mon[33492]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:12 smithi043 ceph-mon[29879]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:12 smithi043 ceph-mon[41444]: pgmap v1512: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:13.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:13.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:14 smithi043 ceph-mon[29879]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:14 smithi043 ceph-mon[41444]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:14 smithi050 ceph-mon[33492]: pgmap v1513: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:16 smithi043 ceph-mon[29879]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:16 smithi043 ceph-mon[41444]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:16 smithi050 ceph-mon[33492]: pgmap v1514: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:18.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:18 smithi043 ceph-mon[29879]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:18 smithi043 ceph-mon[41444]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:18 smithi050 ceph-mon[33492]: pgmap v1515: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:19.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:02:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:02:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:20 smithi043 ceph-mon[29879]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:20 smithi043 ceph-mon[41444]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:20 smithi050 ceph-mon[33492]: pgmap v1516: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:22.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:02:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:02:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:22 smithi043 ceph-mon[29879]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:22 smithi043 ceph-mon[41444]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:22 smithi050 ceph-mon[33492]: pgmap v1517: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:24 smithi043 ceph-mon[29879]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:24 smithi043 ceph-mon[41444]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:24 smithi050 ceph-mon[33492]: pgmap v1518: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:26 smithi043 ceph-mon[41444]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:26 smithi043 ceph-mon[29879]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:26 smithi050 ceph-mon[33492]: pgmap v1519: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:28.625 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:28 smithi050 ceph-mon[33492]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:28 smithi043 ceph-mon[41444]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:28 smithi043 ceph-mon[29879]: pgmap v1520: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:02:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:02:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:30.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:30 smithi043 ceph-mon[29879]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:30.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:30 smithi043 ceph-mon[41444]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:30 smithi050 ceph-mon[33492]: pgmap v1521: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:32.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:02:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:02:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:32 smithi043 ceph-mon[29879]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:32 smithi043 ceph-mon[41444]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:32.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:32 smithi050 ceph-mon[33492]: pgmap v1522: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:34 smithi043 ceph-mon[29879]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:34 smithi043 ceph-mon[41444]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:34 smithi050 ceph-mon[33492]: pgmap v1523: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:36 smithi043 ceph-mon[29879]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:36 smithi043 ceph-mon[41444]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:36 smithi050 ceph-mon[33492]: pgmap v1524: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:38 smithi043 ceph-mon[29879]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:38 smithi043 ceph-mon[41444]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:38 smithi050 ceph-mon[33492]: pgmap v1525: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:02:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:02:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:40 smithi043 ceph-mon[41444]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:40 smithi043 ceph-mon[29879]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:40 smithi050 ceph-mon[33492]: pgmap v1526: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:42.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:02:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:02:42.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:42 smithi043 ceph-mon[29879]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:42 smithi043 ceph-mon[41444]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:42.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:42 smithi050 ceph-mon[33492]: pgmap v1527: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:43.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.605 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.606 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.607 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.608 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:02:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:44 smithi043 ceph-mon[29879]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:44 smithi043 ceph-mon[41444]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:44 smithi050 ceph-mon[33492]: pgmap v1528: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:02:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:45.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:45 smithi043 ceph-mon[29879]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:45 smithi043 ceph-mon[41444]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:45 smithi050 ceph-mon[33492]: pgmap v1529: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:47.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:47 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:02:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:47 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:02:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:47 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:02:48.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:48 smithi043 ceph-mon[29879]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:48.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:48 smithi043 ceph-mon[41444]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:48 smithi050 ceph-mon[33492]: pgmap v1530: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:49.251 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:02:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:02:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:50.452 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:50 smithi043 ceph-mon[41444]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:50.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:50 smithi043 ceph-mon[29879]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:50 smithi050 ceph-mon[33492]: pgmap v1531: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:52.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[29879]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:02:52.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:02:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.314 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:51 smithi043 ceph-mon[41444]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:52.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:02:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:02:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:51 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:51 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:51 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:02:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:51 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:02:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:51 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:02:52.323 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:51 smithi050 ceph-mon[33492]: pgmap v1532: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:53.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:52 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:53.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:52 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:53.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:52 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:02:54.313 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:53 smithi043 ceph-mon[29879]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:54.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:53 smithi043 ceph-mon[41444]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:54.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:53 smithi050 ceph-mon[33492]: pgmap v1533: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:56.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:56 smithi043 ceph-mon[41444]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:56.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:56 smithi043 ceph-mon[29879]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:56.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:56 smithi050 ceph-mon[33492]: pgmap v1534: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:02:58.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:58 smithi043 ceph-mon[41444]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:58 smithi043 ceph-mon[29879]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:58.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:58 smithi050 ceph-mon[33492]: pgmap v1535: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:02:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:02:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:02:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:02:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:02:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:02:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:02:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:02:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:00.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:00 smithi043 ceph-mon[29879]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:00 smithi043 ceph-mon[41444]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:00 smithi050 ceph-mon[33492]: pgmap v1536: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:02.338 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:03:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:03:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:02 smithi043 ceph-mon[29879]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:02 smithi043 ceph-mon[41444]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:02 smithi050 ceph-mon[33492]: pgmap v1537: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:04 smithi043 ceph-mon[29879]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:04 smithi043 ceph-mon[41444]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:04 smithi050 ceph-mon[33492]: pgmap v1538: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:06.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:06 smithi043 ceph-mon[29879]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:06 smithi043 ceph-mon[41444]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:06 smithi050 ceph-mon[33492]: pgmap v1539: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:08.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:08 smithi043 ceph-mon[29879]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:08.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:08 smithi043 ceph-mon[41444]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:08 smithi050 ceph-mon[33492]: pgmap v1540: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:09.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:03:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:03:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:10 smithi043 ceph-mon[29879]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:10 smithi043 ceph-mon[41444]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:10 smithi050 ceph-mon[33492]: pgmap v1541: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:12.357 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:03:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:03:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:12 smithi043 ceph-mon[29879]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:12 smithi043 ceph-mon[41444]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:12 smithi050 ceph-mon[33492]: pgmap v1542: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:13.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:14 smithi043 ceph-mon[29879]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:14.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:14 smithi043 ceph-mon[41444]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:14 smithi050 ceph-mon[33492]: pgmap v1543: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:16 smithi043 ceph-mon[29879]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:16 smithi043 ceph-mon[41444]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:16 smithi050 ceph-mon[33492]: pgmap v1544: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:18.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:18 smithi043 ceph-mon[29879]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:18 smithi043 ceph-mon[41444]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:18 smithi050 ceph-mon[33492]: pgmap v1545: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:03:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:03:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:20.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:20 smithi043 ceph-mon[29879]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:20 smithi043 ceph-mon[41444]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:20 smithi050 ceph-mon[33492]: pgmap v1546: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:22.374 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:03:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:03:22.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:22 smithi043 ceph-mon[29879]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:22 smithi043 ceph-mon[41444]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:22.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:22 smithi050 ceph-mon[33492]: pgmap v1547: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:24.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:24 smithi043 ceph-mon[29879]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:24 smithi043 ceph-mon[41444]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:24 smithi050 ceph-mon[33492]: pgmap v1548: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:26.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:26 smithi043 ceph-mon[29879]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:26 smithi043 ceph-mon[41444]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:26 smithi050 ceph-mon[33492]: pgmap v1549: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:28.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:28 smithi043 ceph-mon[29879]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:28 smithi043 ceph-mon[41444]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:28 smithi050 ceph-mon[33492]: pgmap v1550: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:29.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:03:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:03:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:30.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:30 smithi043 ceph-mon[29879]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:30.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:30 smithi043 ceph-mon[41444]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:30 smithi050 ceph-mon[33492]: pgmap v1551: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:32 smithi050 ceph-mon[33492]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:32.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:03:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:03:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:32 smithi043 ceph-mon[29879]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:32 smithi043 ceph-mon[41444]: pgmap v1552: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:34 smithi043 ceph-mon[29879]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:34 smithi043 ceph-mon[41444]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:34 smithi050 ceph-mon[33492]: pgmap v1553: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:36 smithi043 ceph-mon[29879]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:36 smithi043 ceph-mon[41444]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:36 smithi050 ceph-mon[33492]: pgmap v1554: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:38.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:38 smithi043 ceph-mon[29879]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:38.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:38 smithi043 ceph-mon[41444]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:38 smithi050 ceph-mon[33492]: pgmap v1555: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:03:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:03:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:40.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:40 smithi043 ceph-mon[41444]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:40 smithi043 ceph-mon[29879]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:40 smithi050 ceph-mon[33492]: pgmap v1556: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:42 smithi050 ceph-mon[33492]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:42.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:03:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:03:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:42 smithi043 ceph-mon[29879]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:42 smithi043 ceph-mon[41444]: pgmap v1557: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:44 smithi043 ceph-mon[29879]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:44 smithi043 ceph-mon[41444]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:44 smithi050 ceph-mon[33492]: pgmap v1558: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:46.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:46 smithi043 ceph-mon[41444]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:46.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:46 smithi043 ceph-mon[29879]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:46 smithi050 ceph-mon[33492]: pgmap v1559: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:48.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:48 smithi043 ceph-mon[29879]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:48 smithi043 ceph-mon[41444]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:48 smithi050 ceph-mon[33492]: pgmap v1560: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:03:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:03:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:50 smithi043 ceph-mon[29879]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:50 smithi043 ceph-mon[41444]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:50 smithi050 ceph-mon[33492]: pgmap v1561: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:03:51.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:51 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:03:51.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:51 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:03:51.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:51 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:03:52.412 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:52 smithi050 ceph-mon[33492]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:52.412 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:03:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:03:52.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:52 smithi043 ceph-mon[29879]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:52.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:52 smithi043 ceph-mon[41444]: pgmap v1562: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:53.541 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:53.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:53.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:03:54.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:54 smithi050 ceph-mon[33492]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:54.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:54 smithi043 ceph-mon[29879]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:54.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:54 smithi043 ceph-mon[41444]: pgmap v1563: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:56.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[29879]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[41444]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:03:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:56 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:56 smithi050 ceph-mon[33492]: pgmap v1564: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:03:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:56 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:56 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:56 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:03:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:56 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:03:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:56 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:03:58.649 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:58 smithi050 ceph-mon[33492]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:58 smithi043 ceph-mon[29879]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:58 smithi043 ceph-mon[41444]: pgmap v1565: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:03:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:03:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:03:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:03:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:03:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:03:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:03:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:03:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:00.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:00 smithi043 ceph-mon[29879]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:00 smithi043 ceph-mon[41444]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:00 smithi050 ceph-mon[33492]: pgmap v1566: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:02 smithi050 ceph-mon[33492]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:02.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:04:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:04:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:02 smithi043 ceph-mon[29879]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:02.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:02 smithi043 ceph-mon[41444]: pgmap v1567: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:04 smithi043 ceph-mon[29879]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:04.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:04 smithi043 ceph-mon[41444]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:04 smithi050 ceph-mon[33492]: pgmap v1568: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:06 smithi043 ceph-mon[29879]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:06 smithi043 ceph-mon[41444]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:06 smithi050 ceph-mon[33492]: pgmap v1569: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:08.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:08 smithi043 ceph-mon[29879]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-05T08:04:08.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:08 smithi043 ceph-mon[41444]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-05T08:04:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:08 smithi050 ceph-mon[33492]: pgmap v1570: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-05T08:04:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:04:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:04:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:10 smithi043 ceph-mon[29879]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-05T08:04:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:10 smithi043 ceph-mon[41444]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-05T08:04:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:10 smithi050 ceph-mon[33492]: pgmap v1571: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 4.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-05T08:04:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:12 smithi050 ceph-mon[33492]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:12.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:04:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:04:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:12 smithi043 ceph-mon[29879]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:12 smithi043 ceph-mon[41444]: pgmap v1572: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:13.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:13.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:14 smithi043 ceph-mon[29879]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:14 smithi043 ceph-mon[41444]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:14 smithi050 ceph-mon[33492]: pgmap v1573: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:16 smithi043 ceph-mon[29879]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:16 smithi043 ceph-mon[41444]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:16 smithi050 ceph-mon[33492]: pgmap v1574: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:18.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:18 smithi043 ceph-mon[41444]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:18 smithi043 ceph-mon[29879]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:18 smithi050 ceph-mon[33492]: pgmap v1575: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-05T08:04:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:04:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:04:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:20.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:20 smithi043 ceph-mon[29879]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-02-05T08:04:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:20 smithi043 ceph-mon[41444]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-02-05T08:04:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:20 smithi050 ceph-mon[33492]: pgmap v1576: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 53 op/s 2024-02-05T08:04:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:22 smithi050 ceph-mon[33492]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-02-05T08:04:22.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:04:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:04:22.613 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.613 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.613 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.613 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.613 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.613 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.614 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.621 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.621 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.621 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.621 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:04:22.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:22 smithi043 ceph-mon[29879]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-02-05T08:04:22.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:22 smithi043 ceph-mon[41444]: pgmap v1577: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 33 KiB/s rd, 0 B/s wr, 54 op/s 2024-02-05T08:04:23.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:23.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:24.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:24 smithi043 ceph-mon[29879]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:24 smithi043 ceph-mon[41444]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:24 smithi050 ceph-mon[33492]: pgmap v1578: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:26 smithi043 ceph-mon[29879]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:26 smithi043 ceph-mon[41444]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:26 smithi050 ceph-mon[33492]: pgmap v1579: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:28 smithi043 ceph-mon[29879]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:28 smithi043 ceph-mon[41444]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:28 smithi050 ceph-mon[33492]: pgmap v1580: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:29.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:04:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:04:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:30.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:30 smithi043 ceph-mon[29879]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:30.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:30 smithi043 ceph-mon[41444]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:30 smithi050 ceph-mon[33492]: pgmap v1581: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:32 smithi050 ceph-mon[33492]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:32.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:04:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:04:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:32 smithi043 ceph-mon[29879]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:32.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:32 smithi043 ceph-mon[41444]: pgmap v1582: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:33.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:34 smithi043 ceph-mon[29879]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:34 smithi043 ceph-mon[41444]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:34 smithi050 ceph-mon[33492]: pgmap v1583: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:36 smithi043 ceph-mon[29879]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:36 smithi043 ceph-mon[41444]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:36 smithi050 ceph-mon[33492]: pgmap v1584: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:38.615 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:38 smithi043 ceph-mon[29879]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:38.615 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:38 smithi043 ceph-mon[41444]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:38 smithi050 ceph-mon[33492]: pgmap v1585: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:39.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:04:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:04:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:40.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:40 smithi043 ceph-mon[41444]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:40.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:40 smithi043 ceph-mon[29879]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:40 smithi050 ceph-mon[33492]: pgmap v1586: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:42 smithi050 ceph-mon[33492]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:42.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:04:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:04:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:42 smithi043 ceph-mon[29879]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:42 smithi043 ceph-mon[41444]: pgmap v1587: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:43.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:44.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:44 smithi043 ceph-mon[41444]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:44.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:44 smithi043 ceph-mon[29879]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:44 smithi050 ceph-mon[33492]: pgmap v1588: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:46.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:46 smithi043 ceph-mon[29879]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:46.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:46 smithi043 ceph-mon[41444]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:46 smithi050 ceph-mon[33492]: pgmap v1589: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:48.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:48 smithi043 ceph-mon[29879]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:48 smithi043 ceph-mon[41444]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:48 smithi050 ceph-mon[33492]: pgmap v1590: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:04:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:04:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:50.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:50 smithi043 ceph-mon[29879]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:50 smithi043 ceph-mon[41444]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:50 smithi050 ceph-mon[33492]: pgmap v1591: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:52 smithi050 ceph-mon[33492]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:52.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:04:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:04:52.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:52 smithi043 ceph-mon[29879]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:52.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:52 smithi043 ceph-mon[41444]: pgmap v1592: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:53.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:53.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:53.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:04:54.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:54 smithi043 ceph-mon[29879]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:54.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:54 smithi043 ceph-mon[41444]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:54.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:54 smithi050 ceph-mon[33492]: pgmap v1593: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:04:56.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:56 smithi043 ceph-mon[29879]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:56.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:56 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:04:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:56 smithi043 ceph-mon[41444]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:56 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:04:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:56 smithi050 ceph-mon[33492]: pgmap v1594: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:04:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:56 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:04:58.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[29879]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:57 smithi043 ceph-mon[41444]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:58.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:57 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:57 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:57 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:04:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:57 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:04:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:57 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:04:58.073 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:57 smithi050 ceph-mon[33492]: pgmap v1595: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:04:59.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:59.313 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:04:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:04:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:04:59.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:04:59.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:00.312 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:04:59 smithi043 ceph-mon[29879]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:00.313 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:04:59 smithi043 ceph-mon[41444]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:00.322 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:04:59 smithi050 ceph-mon[33492]: pgmap v1596: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:02 smithi050 ceph-mon[33492]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:02.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:05:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:05:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:02 smithi043 ceph-mon[29879]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:02.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:02 smithi043 ceph-mon[41444]: pgmap v1597: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:04 smithi043 ceph-mon[29879]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:04 smithi043 ceph-mon[41444]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:04 smithi050 ceph-mon[33492]: pgmap v1598: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:06 smithi043 ceph-mon[29879]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:06 smithi043 ceph-mon[41444]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:06 smithi050 ceph-mon[33492]: pgmap v1599: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:08.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:08 smithi043 ceph-mon[29879]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:08.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:08 smithi043 ceph-mon[41444]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:08 smithi050 ceph-mon[33492]: pgmap v1600: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:05:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:05:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:10 smithi043 ceph-mon[29879]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:10 smithi043 ceph-mon[41444]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:10 smithi050 ceph-mon[33492]: pgmap v1601: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:12 smithi050 ceph-mon[33492]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:12.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:05:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:05:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:12 smithi043 ceph-mon[29879]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:12.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:12 smithi043 ceph-mon[41444]: pgmap v1602: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:13.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:13.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:14 smithi043 ceph-mon[29879]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:14 smithi043 ceph-mon[41444]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:14 smithi050 ceph-mon[33492]: pgmap v1603: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:16 smithi043 ceph-mon[29879]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:16 smithi043 ceph-mon[41444]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:16 smithi050 ceph-mon[33492]: pgmap v1604: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:18.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:18 smithi043 ceph-mon[29879]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:18.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:18 smithi043 ceph-mon[41444]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:18 smithi050 ceph-mon[33492]: pgmap v1605: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:05:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:05:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:20.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:20 smithi043 ceph-mon[29879]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:20 smithi043 ceph-mon[41444]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:20 smithi050 ceph-mon[33492]: pgmap v1606: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:22 smithi050 ceph-mon[33492]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:22.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:05:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:05:22.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:22 smithi043 ceph-mon[29879]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:22.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:22 smithi043 ceph-mon[41444]: pgmap v1607: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:23.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:24.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:24 smithi043 ceph-mon[29879]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:24.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:24 smithi043 ceph-mon[41444]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:24 smithi050 ceph-mon[33492]: pgmap v1608: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:26 smithi043 ceph-mon[29879]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:26 smithi043 ceph-mon[41444]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:26 smithi050 ceph-mon[33492]: pgmap v1609: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:28.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:28 smithi043 ceph-mon[41444]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:28.815 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:28 smithi043 ceph-mon[29879]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:28 smithi050 ceph-mon[33492]: pgmap v1610: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:29.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:29.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:05:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:05:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:30.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:30 smithi043 ceph-mon[29879]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:30.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:30 smithi043 ceph-mon[41444]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:30 smithi050 ceph-mon[33492]: pgmap v1611: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:32 smithi050 ceph-mon[33492]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:32.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:05:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:05:32.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:32 smithi043 ceph-mon[41444]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:32 smithi043 ceph-mon[29879]: pgmap v1612: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:34 smithi043 ceph-mon[29879]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:34.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:34 smithi043 ceph-mon[41444]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:34 smithi050 ceph-mon[33492]: pgmap v1613: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:36 smithi043 ceph-mon[29879]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:36 smithi043 ceph-mon[41444]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:36 smithi050 ceph-mon[33492]: pgmap v1614: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:37.322 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:05:36 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a[66914]: logger=cleanup t=2024-02-05T08:05:36.943949029Z level=info msg="Completed cleanup jobs" duration=4.768387ms 2024-02-05T08:05:38.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:38 smithi043 ceph-mon[29879]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:38 smithi043 ceph-mon[41444]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:38 smithi050 ceph-mon[33492]: pgmap v1615: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:05:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:05:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:40 smithi043 ceph-mon[29879]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:40 smithi043 ceph-mon[41444]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:40 smithi050 ceph-mon[33492]: pgmap v1616: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:42 smithi050 ceph-mon[33492]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:42.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:05:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:05:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:42 smithi043 ceph-mon[29879]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:42.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:42 smithi043 ceph-mon[41444]: pgmap v1617: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:43.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:44.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:44 smithi043 ceph-mon[41444]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:44 smithi043 ceph-mon[29879]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:44.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:44 smithi050 ceph-mon[33492]: pgmap v1618: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:46.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:46 smithi043 ceph-mon[29879]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:46.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:46 smithi043 ceph-mon[41444]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:46 smithi050 ceph-mon[33492]: pgmap v1619: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:48.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:48 smithi043 ceph-mon[29879]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:48 smithi043 ceph-mon[41444]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:48 smithi050 ceph-mon[33492]: pgmap v1620: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:05:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:05:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:50 smithi043 ceph-mon[29879]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:50 smithi043 ceph-mon[41444]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:50 smithi050 ceph-mon[33492]: pgmap v1621: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:52 smithi050 ceph-mon[33492]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:52.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:05:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:05:52.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:52 smithi043 ceph-mon[29879]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:52.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:52 smithi043 ceph-mon[41444]: pgmap v1622: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:53.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:53.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:53.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:05:54.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:54 smithi043 ceph-mon[29879]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:54.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:54 smithi043 ceph-mon[41444]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:54.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:54 smithi050 ceph-mon[33492]: pgmap v1623: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:05:56.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:56 smithi043 ceph-mon[29879]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:56 smithi043 ceph-mon[41444]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:56 smithi050 ceph-mon[33492]: pgmap v1624: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:57 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:05:57.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:57 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:05:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:57 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:05:58.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:58 smithi050 ceph-mon[33492]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:05:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:05:58.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:58 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[29879]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[41444]: pgmap v1625: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:05:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:58 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:05:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:05:59 smithi043 ceph-mon[29879]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:59.563 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:05:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:05:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:05:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:05:59 smithi043 ceph-mon[41444]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:05:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:05:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:05:59 smithi050 ceph-mon[33492]: pgmap v1626: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:02 smithi050 ceph-mon[33492]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:02.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:06:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:02 smithi043 ceph-mon[29879]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:02 smithi043 ceph-mon[41444]: pgmap v1627: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:04 smithi043 ceph-mon[29879]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:04 smithi043 ceph-mon[41444]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:04 smithi050 ceph-mon[33492]: pgmap v1628: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:06 smithi043 ceph-mon[29879]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:06 smithi043 ceph-mon[41444]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:06 smithi050 ceph-mon[33492]: pgmap v1629: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:08.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:08 smithi043 ceph-mon[41444]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:08.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:08 smithi043 ceph-mon[29879]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:08.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:08 smithi050 ceph-mon[33492]: pgmap v1630: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:06:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:06:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:10 smithi043 ceph-mon[29879]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:10.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:10 smithi043 ceph-mon[41444]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:10 smithi050 ceph-mon[33492]: pgmap v1631: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:12 smithi050 ceph-mon[33492]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:12.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:06:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:06:12.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:12 smithi043 ceph-mon[29879]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:12 smithi043 ceph-mon[41444]: pgmap v1632: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:13.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:13.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:14 smithi043 ceph-mon[29879]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:14 smithi043 ceph-mon[41444]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:14 smithi050 ceph-mon[33492]: pgmap v1633: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:16 smithi043 ceph-mon[29879]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:16.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:16 smithi043 ceph-mon[41444]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:16 smithi050 ceph-mon[33492]: pgmap v1634: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:18.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:18 smithi043 ceph-mon[29879]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:18.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:18 smithi043 ceph-mon[41444]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:18 smithi050 ceph-mon[33492]: pgmap v1635: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:06:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:06:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:20.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:20 smithi043 ceph-mon[29879]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:20.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:20 smithi043 ceph-mon[41444]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:20 smithi050 ceph-mon[33492]: pgmap v1636: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:22 smithi050 ceph-mon[33492]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:22.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:06:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:06:22.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:22 smithi043 ceph-mon[29879]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:22.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:22 smithi043 ceph-mon[41444]: pgmap v1637: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:23.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:24.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:24 smithi043 ceph-mon[29879]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:24 smithi043 ceph-mon[41444]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:24 smithi050 ceph-mon[33492]: pgmap v1638: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:26 smithi043 ceph-mon[29879]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:26 smithi043 ceph-mon[41444]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:26 smithi050 ceph-mon[33492]: pgmap v1639: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:28 smithi043 ceph-mon[29879]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:28.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:28 smithi043 ceph-mon[41444]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:28 smithi050 ceph-mon[33492]: pgmap v1640: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:29.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:29.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:29 smithi043 ceph-mon[29879]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:06:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:06:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:29 smithi043 ceph-mon[41444]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:29 smithi050 ceph-mon[33492]: pgmap v1641: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:32 smithi050 ceph-mon[33492]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:32.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:06:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:06:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:32 smithi043 ceph-mon[29879]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:32.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:32 smithi043 ceph-mon[41444]: pgmap v1642: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:33.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:33.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:34.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:34 smithi043 ceph-mon[41444]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:34 smithi043 ceph-mon[29879]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:34 smithi050 ceph-mon[33492]: pgmap v1643: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-05T08:06:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:36 smithi043 ceph-mon[29879]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:36 smithi043 ceph-mon[41444]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:36 smithi050 ceph-mon[33492]: pgmap v1644: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:38.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:38 smithi043 ceph-mon[29879]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:38.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:38 smithi043 ceph-mon[41444]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:38 smithi050 ceph-mon[33492]: pgmap v1645: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:06:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:06:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:40 smithi043 ceph-mon[29879]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:40.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:40 smithi043 ceph-mon[41444]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:40.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:40 smithi050 ceph-mon[33492]: pgmap v1646: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:42 smithi050 ceph-mon[33492]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:42.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:06:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:06:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:42 smithi043 ceph-mon[29879]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:42.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:42 smithi043 ceph-mon[41444]: pgmap v1647: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:43.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:44 smithi043 ceph-mon[29879]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:44 smithi043 ceph-mon[41444]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:44 smithi050 ceph-mon[33492]: pgmap v1648: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:46.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:46 smithi043 ceph-mon[29879]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:46 smithi043 ceph-mon[41444]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:46 smithi050 ceph-mon[33492]: pgmap v1649: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:48.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:48 smithi043 ceph-mon[29879]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:48 smithi043 ceph-mon[41444]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:48 smithi050 ceph-mon[33492]: pgmap v1650: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:06:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:06:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:50 smithi043 ceph-mon[29879]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:50.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:50 smithi043 ceph-mon[41444]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:50 smithi050 ceph-mon[33492]: pgmap v1651: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:52 smithi050 ceph-mon[33492]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:52.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:06:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:06:52.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:52 smithi043 ceph-mon[41444]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:52.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:52 smithi043 ceph-mon[29879]: pgmap v1652: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:06:53.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:53.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:53.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:06:54.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:54 smithi043 ceph-mon[41444]: pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:54.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:54 smithi043 ceph-mon[29879]: pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:54.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:54 smithi050 ceph-mon[33492]: pgmap v1653: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:55.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 512 aios 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567be5e720 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19b680 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19c000 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19bb60 2024-02-05T08:06:55.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19cb20 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19c950 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19d030 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19db70 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19da30 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19dc80 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e140 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19dde0 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19def0 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e250 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e6e0 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e7f0 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19d670 2024-02-05T08:06:55.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19d7e0 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19d8f0 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e900 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19ea50 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f000 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19eb60 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19ec70 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19ed80 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19eeb0 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f5b0 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f110 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f240 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f350 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f4a0 2024-02-05T08:06:55.638 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19fb60 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f6c0 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f7d0 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19f900 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19fa50 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0150 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19fcb0 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19fdc0 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19fed0 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0020 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0720 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0280 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0390 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a04a0 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a05b0 2024-02-05T08:06:55.639 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0d10 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0870 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a09c0 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0ad0 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0c00 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a12c0 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0e20 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a0f30 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1040 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1150 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1890 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a13f0 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1500 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1610 2024-02-05T08:06:55.640 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1720 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1e40 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a19a0 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1ab0 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1bc0 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1d10 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a23f0 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a1f50 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a2060 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a2190 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a22a0 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a29e0 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a2540 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a2670 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a27d0 2024-02-05T08:06:55.641 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a2af0 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a2fa0 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a30b0 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e360 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e490 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a36a0 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3200 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3350 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3460 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3570 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c19e5a0 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3810 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3c80 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3db0 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4360 2024-02-05T08:06:55.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3920 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3a30 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3b60 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a44b0 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a3ec0 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4020 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4150 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a45e0 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4a70 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4b80 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a46f0 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4870 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5150 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4cb0 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4de0 2024-02-05T08:06:55.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a4ef0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5000 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5700 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5260 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a53b0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a54c0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a55f0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5cb0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5810 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5960 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5a70 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5ba0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6280 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5de0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a5ef0 2024-02-05T08:06:55.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6020 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6170 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6850 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a63b0 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a64e0 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6610 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6740 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6e20 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6980 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6ad0 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6c00 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6f50 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a73e0 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7510 2024-02-05T08:06:55.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7ae0 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7080 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a6d10 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7190 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a72a0 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7640 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7750 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7860 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7970 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a80d0 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7c30 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7d60 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7e70 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a7fc0 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a86a0 2024-02-05T08:06:55.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8200 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8330 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8440 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8590 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8c90 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a87f0 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8940 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8a50 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8b80 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a95e0 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a9140 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a9270 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a93a0 2024-02-05T08:06:55.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a9730 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a9c20 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aa0b0 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aa1e0 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8da0 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a8ed0 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a9000 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a9880 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a94d0 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a99b0 2024-02-05T08:06:55.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1a9ac0 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aa310 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aa460 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aaa50 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aa5b0 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aa700 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aa830 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aab80 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ab010 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ab140 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aac90 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aade0 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ab710 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ab270 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ab840 2024-02-05T08:06:55.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1abcd0 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac160 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac290 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac860 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1abe00 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1abf10 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac040 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac9b0 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac3c0 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac510 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac620 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ac750 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1acf80 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1acae0 2024-02-05T08:06:55.650 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1acbf0 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1acd40 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ace70 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad590 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad0f0 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad200 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad330 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad480 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1adb60 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad6c0 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad7f0 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ad900 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ada30 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae130 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1adc90 2024-02-05T08:06:55.651 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1addc0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aded0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae280 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae6f0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae800 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aedf0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae390 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae4e0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae950 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aea80 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aebb0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1aef40 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af0b0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ae000 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af1e0 2024-02-05T08:06:55.652 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af670 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af7c0 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ab950 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1abae0 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1afdb0 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af910 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1afa40 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1afb70 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1afca0 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0360 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1afec0 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1afff0 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0100 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0210 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0930 2024-02-05T08:06:55.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0490 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b05a0 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b06f0 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0820 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0f20 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0a80 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0bd0 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0ce0 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1050 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b14e0 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1630 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b0e10 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1ac0 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1760 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b18b0 2024-02-05T08:06:55.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af2f0 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af400 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1af530 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2070 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1bd0 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1d20 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1e30 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1f60 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2660 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b21c0 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2310 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2420 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b27b0 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2c20 2024-02-05T08:06:55.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2d70 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b1160 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b12b0 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3340 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2ea0 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2ff0 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3100 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3230 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3910 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3470 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b35a0 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b36b0 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3a40 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3eb0 2024-02-05T08:06:55.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4000 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b45b0 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3b50 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b3c80 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4110 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4240 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4350 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4460 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4ba0 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4700 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4850 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4980 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4cf0 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b51a0 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b52d0 2024-02-05T08:06:55.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b28c0 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b29f0 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b58a0 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5400 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5530 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b2b00 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5660 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5790 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b59d0 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5e40 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5fb0 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4e20 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b4f70 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b65a0 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6100 2024-02-05T08:06:55.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6230 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6360 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6490 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6b70 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b66d0 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6820 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6950 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6cc0 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7150 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7280 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5ae0 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b5c10 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7850 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b73b0 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b74e0 2024-02-05T08:06:55.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7610 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7740 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7e40 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b79a0 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7ab0 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7be0 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7d10 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b83f0 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b7f50 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8080 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b81b0 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8540 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b89b0 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8ae0 2024-02-05T08:06:55.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6df0 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b6f20 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9090 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8bf0 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8d20 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8e70 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b91c0 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9650 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9780 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8650 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8760 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b8890 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b98b0 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9d40 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9e90 2024-02-05T08:06:55.661 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b92f0 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9420 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ba460 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9fc0 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ba0f0 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ba220 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ba350 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1baa30 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ba590 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ba6e0 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1ba830 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bab80 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb010 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb140 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b99e0 2024-02-05T08:06:55.662 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9af0 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1b9c20 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb270 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb6e0 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb810 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bacb0 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bae00 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bbdc0 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb920 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bba50 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bbb80 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bbcb0 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc3b0 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bbf10 2024-02-05T08:06:55.663 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc040 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc150 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc2a0 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc980 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc4e0 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc630 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bc780 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bcab0 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bcf40 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd070 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb380 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bb4b0 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd640 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd1a0 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd2d0 2024-02-05T08:06:55.664 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd400 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd530 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bdc10 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd770 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd8c0 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bd9f0 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bdd40 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1be1d0 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1be300 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bdb00 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1be790 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1be8a0 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bea10 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bde50 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bdfa0 2024-02-05T08:06:55.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bcbc0 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bccf0 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bce20 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1beb60 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1beff0 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf140 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1be410 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1be560 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf710 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf270 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf3a0 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf4d0 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf600 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bfce0 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf840 2024-02-05T08:06:55.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bf990 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bfaa0 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bfbb0 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c02b0 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bfe10 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bff60 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c00b0 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0400 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c08e0 2024-02-05T08:06:55.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0a10 2024-02-05T08:06:56.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:56 smithi043 ceph-mon[29879]: pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:56.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:56 smithi043 ceph-mon[41444]: pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:56 smithi050 ceph-mon[33492]: pgmap v1654: 228 pgs: 228 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:57 smithi043 ceph-mon[29879]: osdmap e758: 8 total, 8 up, 8 in 2024-02-05T08:06:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:57 smithi043 ceph-mon[29879]: pgmap v1656: 196 pgs: 196 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:06:57.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:57 smithi043 ceph-mon[29879]: osdmap e759: 8 total, 8 up, 8 in 2024-02-05T08:06:57.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:57 smithi043 ceph-mon[41444]: osdmap e758: 8 total, 8 up, 8 in 2024-02-05T08:06:57.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:57 smithi043 ceph-mon[41444]: pgmap v1656: 196 pgs: 196 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:06:57.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:57 smithi043 ceph-mon[41444]: osdmap e759: 8 total, 8 up, 8 in 2024-02-05T08:06:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:57 smithi050 ceph-mon[33492]: osdmap e758: 8 total, 8 up, 8 in 2024-02-05T08:06:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:57 smithi050 ceph-mon[33492]: pgmap v1656: 196 pgs: 196 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:06:57.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:57 smithi050 ceph-mon[33492]: osdmap e759: 8 total, 8 up, 8 in 2024-02-05T08:06:58.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:58 smithi050 ceph-mon[33492]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:58.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:58 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:06:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:58 smithi043 ceph-mon[29879]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:58.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:58 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:06:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:58 smithi043 ceph-mon[41444]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:06:58.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:58 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:06:59.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:06:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: osdmap e760: 8 total, 8 up, 8 in 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[29879]: pgmap v1659: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: osdmap e760: 8 total, 8 up, 8 in 2024-02-05T08:06:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:06:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:06:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:06:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:06:59 smithi043 ceph-mon[41444]: pgmap v1659: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: osdmap e760: 8 total, 8 up, 8 in 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:06:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:06:59 smithi050 ceph-mon[33492]: pgmap v1659: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 839 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:00.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[29879]: osdmap e761: 8 total, 8 up, 8 in 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[41444]: osdmap e761: 8 total, 8 up, 8 in 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:00 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:00 smithi050 ceph-mon[33492]: osdmap e761: 8 total, 8 up, 8 in 2024-02-05T08:07:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-05T08:07:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:00.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:00 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:01.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[29879]: osdmap e762: 8 total, 8 up, 8 in 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2"}]: dispatch 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[29879]: pgmap v1662: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[41444]: osdmap e762: 8 total, 8 up, 8 in 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2"}]: dispatch 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:01.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:01.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:01 smithi043 ceph-mon[41444]: pgmap v1662: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:07:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:01 smithi050 ceph-mon[33492]: osdmap e762: 8 total, 8 up, 8 in 2024-02-05T08:07:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2"}]: dispatch 2024-02-05T08:07:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:01 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:01 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-05T08:07:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:01 smithi050 ceph-mon[33492]: pgmap v1662: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:07:02.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:07:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:07:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:02 smithi050 ceph-mon[33492]: osdmap e763: 8 total, 8 up, 8 in 2024-02-05T08:07:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-05T08:07:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:07:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:02 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:07:02.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[29879]: osdmap e763: 8 total, 8 up, 8 in 2024-02-05T08:07:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-05T08:07:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81922-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[41444]: osdmap e763: 8 total, 8 up, 8 in 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-05T08:07:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:03.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T08:07:03.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[41444]: osdmap e764: 8 total, 8 up, 8 in 2024-02-05T08:07:03.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:07:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:07:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[41444]: pgmap v1665: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:07:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T08:07:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[29879]: osdmap e764: 8 total, 8 up, 8 in 2024-02-05T08:07:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:07:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:07:03.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:03 smithi043 ceph-mon[29879]: pgmap v1665: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:07:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-05T08:07:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:03 smithi050 ceph-mon[33492]: osdmap e764: 8 total, 8 up, 8 in 2024-02-05T08:07:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:07:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:03 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-05T08:07:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:03 smithi050 ceph-mon[33492]: pgmap v1665: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-05T08:07:04.492 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bec90 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1bedc0 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0fe0 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0b40 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0c70 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0da0 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1110 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c15f0 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1a80 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1bb0 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1240 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1370 2024-02-05T08:07:04.493 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c14a0 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0510 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0640 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0790 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c0ed0 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1ce0 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2170 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c22c0 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c28b0 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1e10 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1f60 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2410 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2540 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2670 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c27a0 2024-02-05T08:07:04.494 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2e80 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c29e0 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2af0 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2c20 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2d50 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3450 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c2fb0 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3100 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3210 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3340 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3a20 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3580 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c36b0 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c37c0 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3910 2024-02-05T08:07:04.495 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4010 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3b70 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3cc0 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3dd0 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4180 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4610 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4760 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c3f00 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4c10 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4d20 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4e50 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4290 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c43a0 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c44f0 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1720 2024-02-05T08:07:04.496 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c1870 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4fa0 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c5410 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c5540 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c48b0 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c4a00 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c5b30 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55567c1c5690 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: done. 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1811610 ms) 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1811610 ms total) 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-05T08:07:04.497 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-05T08:07:04.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Exec (14 ms) 2024-02-05T08:07:04.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-05T08:07:04.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (8 ms) 2024-02-05T08:07:04.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-05T08:07:04.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-02-05T08:07:04.498 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-05T08:07:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T08:07:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[29879]: osdmap e765: 8 total, 8 up, 8 in 2024-02-05T08:07:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:07:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:07:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T08:07:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[41444]: osdmap e765: 8 total, 8 up, 8 in 2024-02-05T08:07:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:07:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:04 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:07:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-05T08:07:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:04 smithi050 ceph-mon[33492]: osdmap e765: 8 total, 8 up, 8 in 2024-02-05T08:07:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/148263015' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:07:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:04 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]: dispatch 2024-02-05T08:07:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:05 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]': finished 2024-02-05T08:07:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:05 smithi043 ceph-mon[29879]: osdmap e766: 8 total, 8 up, 8 in 2024-02-05T08:07:05.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:05 smithi043 ceph-mon[29879]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:05 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]': finished 2024-02-05T08:07:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:05 smithi043 ceph-mon[41444]: osdmap e766: 8 total, 8 up, 8 in 2024-02-05T08:07:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:05 smithi043 ceph-mon[41444]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:05 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81922-1","app":"app1","key":"key1"}]': finished 2024-02-05T08:07:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:05 smithi050 ceph-mon[33492]: osdmap e766: 8 total, 8 up, 8 in 2024-02-05T08:07:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:05 smithi050 ceph-mon[33492]: pgmap v1668: 196 pgs: 196 active+clean; 583 KiB data, 842 MiB used, 714 GiB / 715 GiB avail 2024-02-05T08:07:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:06 smithi043 ceph-mon[29879]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:06.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:06 smithi043 ceph-mon[41444]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:06.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:06 smithi050 ceph-mon[33492]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:07.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:07 smithi043 ceph-mon[41444]: pgmap v1669: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-02-05T08:07:07.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:07 smithi043 ceph-mon[29879]: pgmap v1669: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-02-05T08:07:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:07 smithi050 ceph-mon[33492]: pgmap v1669: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 877 B/s rd, 0 op/s 2024-02-05T08:07:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:07:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:07:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:10 smithi043 ceph-mon[29879]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-02-05T08:07:10.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:10 smithi043 ceph-mon[41444]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-02-05T08:07:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:10 smithi050 ceph-mon[33492]: pgmap v1670: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 749 B/s rd, 0 op/s 2024-02-05T08:07:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:12 smithi050 ceph-mon[33492]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:12.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:07:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:07:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:12 smithi043 ceph-mon[29879]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:12.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:12 smithi043 ceph-mon[41444]: pgmap v1671: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:13.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:13.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:13.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:14 smithi043 ceph-mon[29879]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:07:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:14 smithi043 ceph-mon[41444]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:07:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:14.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:14 smithi050 ceph-mon[33492]: pgmap v1672: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:07:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:16 smithi043 ceph-mon[29879]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-02-05T08:07:16.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:16 smithi043 ceph-mon[41444]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-02-05T08:07:16.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:16 smithi050 ceph-mon[33492]: pgmap v1673: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 946 B/s rd, 0 op/s 2024-02-05T08:07:18.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:18 smithi043 ceph-mon[29879]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:18 smithi043 ceph-mon[41444]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:18.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:18 smithi050 ceph-mon[33492]: pgmap v1674: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:07:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:07:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:20.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:20 smithi043 ceph-mon[29879]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:20.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:20 smithi043 ceph-mon[41444]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:20.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:20 smithi050 ceph-mon[33492]: pgmap v1675: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5014 ms) 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (17883 ms) 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc (22922 ms total) 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment tear-down 2024-02-05T08:07:22.426 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1838661 ms total) 2024-02-05T08:07:22.427 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-05T08:07:22.432 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.432 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82028 2024-02-05T08:07:22.432 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82028 2024-02-05T08:07:22.432 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.433 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82671 2024-02-05T08:07:22.433 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82671 2024-02-05T08:07:22.433 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.433 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82720 2024-02-05T08:07:22.433 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82720 2024-02-05T08:07:22.433 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.433 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81774 2024-02-05T08:07:22.434 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81774 2024-02-05T08:07:22.434 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.434 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82495 2024-02-05T08:07:22.434 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82495 2024-02-05T08:07:22.434 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.434 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82356 2024-02-05T08:07:22.434 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82356 2024-02-05T08:07:22.435 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.435 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81643 2024-02-05T08:07:22.435 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81643 2024-02-05T08:07:22.435 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.435 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81707 2024-02-05T08:07:22.435 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81707 2024-02-05T08:07:22.435 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.436 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82383 2024-02-05T08:07:22.436 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82383 2024-02-05T08:07:22.436 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.436 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81634 2024-02-05T08:07:22.436 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81634 2024-02-05T08:07:22.436 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.436 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81656 2024-02-05T08:07:22.437 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81656 2024-02-05T08:07:22.437 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.437 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82806 2024-02-05T08:07:22.437 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82806 2024-02-05T08:07:22.437 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.438 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81865 2024-02-05T08:07:22.438 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81865 2024-02-05T08:07:22.438 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.438 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82873 2024-02-05T08:07:22.438 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82873 2024-02-05T08:07:22.438 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.439 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82016 2024-02-05T08:07:22.439 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82016 2024-02-05T08:07:22.439 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.439 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82614 2024-02-05T08:07:22.439 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82614 2024-02-05T08:07:22.439 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.439 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82926 2024-02-05T08:07:22.440 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82926 2024-02-05T08:07:22.440 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.440 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81914 2024-02-05T08:07:22.440 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81914 2024-02-05T08:07:22.440 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.440 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82410 2024-02-05T08:07:22.440 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82410 2024-02-05T08:07:22.441 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.441 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82788 2024-02-05T08:07:22.441 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82788 2024-02-05T08:07:22.441 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.441 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82557 2024-02-05T08:07:22.441 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82557 2024-02-05T08:07:22.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82826 2024-02-05T08:07:22.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82826 2024-02-05T08:07:22.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82349 2024-02-05T08:07:22.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82349 2024-02-05T08:07:22.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81830 2024-02-05T08:07:22.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81830 2024-02-05T08:07:22.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82898 2024-02-05T08:07:22.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82898 2024-02-05T08:07:22.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.443 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82153 2024-02-05T08:07:22.444 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82153 2024-02-05T08:07:22.444 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.444 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81627 2024-02-05T08:07:22.444 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81627 2024-02-05T08:07:22.444 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.444 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81686 2024-02-05T08:07:22.444 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81686 2024-02-05T08:07:22.445 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.445 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82075 2024-02-05T08:07:22.445 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82075 2024-02-05T08:07:22.445 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.445 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81727 2024-02-05T08:07:22.445 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81727 2024-02-05T08:07:22.446 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.446 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82201 2024-02-05T08:07:22.446 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82201 2024-02-05T08:07:22.446 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.446 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82281 2024-02-05T08:07:22.446 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82281 2024-02-05T08:07:22.446 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.447 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82773 2024-02-05T08:07:22.447 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82773 2024-02-05T08:07:22.447 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-02-05T08:07:22.447 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82976 2024-02-05T08:07:22.447 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82976 2024-02-05T08:07:22.447 INFO:tasks.workunit.client.0.smithi043.stderr:+ exit 0 2024-02-05T08:07:22.447 INFO:tasks.workunit.client.0.smithi043.stderr:+ cleanup 2024-02-05T08:07:22.448 INFO:tasks.workunit.client.0.smithi043.stderr:+ pkill -P 81622 2024-02-05T08:07:22.450 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-02-05T08:07:22.450 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-05T08:07:22.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-05T08:07:22.481 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-05T08:07:22.482 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-05T08:07:22.483 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=164d3be4a527be775f4d3c343ea591122f1b8921 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-05T08:07:22.539 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-02-05T08:07:22.546 INFO:tasks.workunit.client.0.smithi043.stderr:+ p=e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:07:22.546 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create e2a6ede4-caf1-4034-97c2-ddac1835f21e 12 2024-02-05T08:07:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:22 smithi050 ceph-mon[33492]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:22.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:07:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:07:22.632 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2084523517 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 msgr2=0x7f9c8c10e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:22.632 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2084523517 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c10e170 secure :-1 s=READY pgs=3143 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009c10 tx=0x7f9c8402bef0 comp rx=0 tx=0).stop 2024-02-05T08:07:22.632 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2084523517 shutdown_connections 2024-02-05T08:07:22.632 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2084523517 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c10e170 unknown :-1 s=CLOSED pgs=3143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:22.632 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2084523517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8c0ffbd0 0x7f9c8c100050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:22.632 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2084523517 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c1062d0 0x7f9c8c1066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:22.632 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2084523517 >> 172.21.15.43:0/2084523517 conn(0x7f9c8c0754b0 msgr2=0x7f9c8c0758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2084523517 shutdown_connections 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2084523517 wait complete. 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 Processor -- start 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.629+0000 7f9c93d6c640 1 -- start start 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c93d6c640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0ffbd0 0x7f9c8c196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c93d6c640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c1972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c93d6c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8c1062d0 0x7f9c8c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c93d6c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c19be30 con 0x7f9c8c1062d0 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c93d6c640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c19bfa0 con 0x7f9c8c0ffbd0 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c93d6c640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8c19c160 con 0x7f9c8c100590 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c1972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:22.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c1972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48690/0 (socket says 172.21.15.43:48690) 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 -- 172.21.15.43:0/2474495787 learned_addr learned my addr 172.21.15.43:0/2474495787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c92309640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8c1062d0 0x7f9c8c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91b08640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0ffbd0 0x7f9c8c196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 -- 172.21.15.43:0/2474495787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0ffbd0 msgr2=0x7f9c8c196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0ffbd0 0x7f9c8c196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 -- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8c1062d0 msgr2=0x7f9c8c19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8c1062d0 0x7f9c8c19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 -- 172.21.15.43:0/2474495787 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7c009560 con 0x7f9c8c100590 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91b08640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0ffbd0 0x7f9c8c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c91307640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c1972f0 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e9c0 tx=0x7f9c7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c00cde0 con 0x7f9c8c100590 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f9c7c0044a0 con 0x7f9c8c100590 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c84009840 con 0x7f9c8c100590 2024-02-05T08:07:22.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.630+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c010430 con 0x7f9c8c100590 2024-02-05T08:07:22.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.631+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8c1a3ae0 con 0x7f9c8c100590 2024-02-05T08:07:22.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.631+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8c0760e0 con 0x7f9c8c100590 2024-02-05T08:07:22.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.632+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f9c7c003ee0 con 0x7f9c8c100590 2024-02-05T08:07:22.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.632+0000 7f9c82ffd640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f9c7806da30 0x7f9c7806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:22.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.632+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(767..767 src has 257..767) v4 ==== 8679+0+0 (secure 0 0 0) 0x7f9c7c014070 con 0x7f9c8c100590 2024-02-05T08:07:22.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.632+0000 7f9c91b08640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f9c7806da30 0x7f9c7806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:22.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.633+0000 7f9c91b08640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f9c7806da30 0x7f9c7806fef0 secure :-1 s=READY pgs=4059 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b40 tx=0x7f9c88005e50 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:22.637 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.637+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c7c057170 con 0x7f9c8c100590 2024-02-05T08:07:22.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:22 smithi043 ceph-mon[29879]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:22.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:22 smithi043 ceph-mon[41444]: pgmap v1676: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:22.823 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:22.822+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12} v 0) v1 -- 0x7f9c8c04f070 con 0x7f9c8c100590 2024-02-05T08:07:23.417 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.417+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]=0 pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' created v768) v1 ==== 176+0+0 (secure 0 0 0) 0x7f9c7c05a830 con 0x7f9c8c100590 2024-02-05T08:07:23.532 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.532+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12} v 0) v1 -- 0x7f9c8c110840 con 0x7f9c8c100590 2024-02-05T08:07:23.533 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.533+0000 7f9c82ffd640 1 -- 172.21.15.43:0/2474495787 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]=0 pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' already exists v768) v1 ==== 183+0+0 (secure 0 0 0) 0x7f9c7c05aa10 con 0x7f9c8c100590 2024-02-05T08:07:23.533 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' already exists 2024-02-05T08:07:23.535 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f9c7806da30 msgr2=0x7f9c7806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:23.535 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f9c7806da30 0x7f9c7806fef0 secure :-1 s=READY pgs=4059 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009b40 tx=0x7f9c88005e50 comp rx=0 tx=0).stop 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 msgr2=0x7f9c8c1972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c1972f0 secure :-1 s=READY pgs=3144 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c00e9c0 tx=0x7f9c7c00ee80 comp rx=0 tx=0).stop 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 shutdown_connections 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f9c7806da30 0x7f9c7806fef0 unknown :-1 s=CLOSED pgs=4059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c8c1062d0 0x7f9c8c19b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f9c8c100590 0x7f9c8c1972f0 unknown :-1 s=CLOSED pgs=3144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 --2- 172.21.15.43:0/2474495787 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f9c8c0ffbd0 0x7f9c8c196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 >> 172.21.15.43:0/2474495787 conn(0x7f9c8c0754b0 msgr2=0x7f9c8c0fd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 shutdown_connections 2024-02-05T08:07:23.536 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.535+0000 7f9c93d6c640 1 -- 172.21.15.43:0/2474495787 wait complete. 2024-02-05T08:07:23.545 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota e2a6ede4-caf1-4034-97c2-ddac1835f21e max_objects 10 2024-02-05T08:07:23.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 -- 172.21.15.43:0/2512713908 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc1065d0 msgr2=0x7f31bc10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:23.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/2512713908 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc1065d0 0x7f31bc10e1a0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009900 tx=0x7f31a402bc30 comp rx=0 tx=0).stop 2024-02-05T08:07:23.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 -- 172.21.15.43:0/2512713908 shutdown_connections 2024-02-05T08:07:23.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/2512713908 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc1065d0 0x7f31bc10e1a0 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.633 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/2512713908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101eb0 0x7f31bc105ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/2512713908 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31bc101500 0x7f31bc1018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 -- 172.21.15.43:0/2512713908 >> 172.21.15.43:0/2512713908 conn(0x7f31bc0fcd60 msgr2=0x7f31bc0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 -- 172.21.15.43:0/2512713908 shutdown_connections 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.628+0000 7f31c2d7a640 1 -- 172.21.15.43:0/2512713908 wait complete. 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 Processor -- start 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 -- start start 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101500 0x7f31bc196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31bc1065d0 0x7f31bc197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc19b920 0x7f31bc19bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc19c310 con 0x7f31bc101500 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc19c480 con 0x7f31bc1065d0 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c2d7a640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc19c5f0 con 0x7f31bc19b920 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c0b16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101500 0x7f31bc196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:23.634 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c1317640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc19b920 0x7f31bc19bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c1317640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc19b920 0x7f31bc19bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48702/0 (socket says 172.21.15.43:48702) 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c0b16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101500 0x7f31bc196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38674/0 (socket says 172.21.15.43:38674) 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31c1317640 1 -- 172.21.15.43:0/4232727246 learned_addr learned my addr 172.21.15.43:0/4232727246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.629+0000 7f31bbfff640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31bc1065d0 0x7f31bc197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c1317640 1 -- 172.21.15.43:0/4232727246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31bc1065d0 msgr2=0x7f31bc197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c1317640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31bc1065d0 0x7f31bc197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c1317640 1 -- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101500 msgr2=0x7f31bc196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:23.635 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c1317640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101500 0x7f31bc196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c1317640 1 -- 172.21.15.43:0/4232727246 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac009490 con 0x7f31bc19b920 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c0b16640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101500 0x7f31bc196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c1317640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc19b920 0x7f31bc19bdd0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f31a4005940 tx=0x7f31a40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a40044e0 con 0x7f31bc19b920 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f31a4003c80 con 0x7f31bc19b920 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31a4009560 con 0x7f31bc19b920 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4005470 con 0x7f31bc19b920 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.630+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc1a3ef0 con 0x7f31bc19b920 2024-02-05T08:07:23.636 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.631+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc1029d0 con 0x7f31bc19b920 2024-02-05T08:07:23.637 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.632+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f31a4003800 con 0x7f31bc19b920 2024-02-05T08:07:23.637 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.632+0000 7f31b9ffb640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f31a806da30 0x7f31a806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:23.637 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.632+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(768..768 src has 257..768) v4 ==== 9054+0+0 (secure 0 0 0) 0x7f31a40b0210 con 0x7f31bc19b920 2024-02-05T08:07:23.637 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.632+0000 7f31c0b16640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f31a806da30 0x7f31a806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:23.637 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.633+0000 7f31c0b16640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f31a806da30 0x7f31a806fef0 secure :-1 s=READY pgs=4060 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005c40 tx=0x7f31b400a570 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:23.637 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.637+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31a4079100 con 0x7f31bc19b920 2024-02-05T08:07:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[29879]: osdmap e767: 8 total, 8 up, 8 in 2024-02-05T08:07:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2474495787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[41444]: osdmap e767: 8 total, 8 up, 8 in 2024-02-05T08:07:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2474495787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:23.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:23 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:23.819 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:23.818+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f31bc1983b0 con 0x7f31bc19b920 2024-02-05T08:07:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:23 smithi050 ceph-mon[33492]: osdmap e767: 8 total, 8 up, 8 in 2024-02-05T08:07:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:23 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2474495787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:23 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:24.419 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:24.418+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v769) v1 ==== 223+0+0 (secure 0 0 0) 0x7f31a407c7c0 con 0x7f31bc19b920 2024-02-05T08:07:24.539 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:24.539+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f31bc101eb0 con 0x7f31bc19b920 2024-02-05T08:07:24.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[41444]: pgmap v1678: 164 pgs: 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-02-05T08:07:24.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]': finished 2024-02-05T08:07:24.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[41444]: osdmap e768: 8 total, 8 up, 8 in 2024-02-05T08:07:24.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2474495787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4232727246' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[29879]: pgmap v1678: 164 pgs: 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]': finished 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[29879]: osdmap e768: 8 total, 8 up, 8 in 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2474495787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4232727246' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:24.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:24 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:24 smithi050 ceph-mon[33492]: pgmap v1678: 164 pgs: 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 818 B/s rd, 0 op/s 2024-02-05T08:07:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]': finished 2024-02-05T08:07:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:24 smithi050 ceph-mon[33492]: osdmap e768: 8 total, 8 up, 8 in 2024-02-05T08:07:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2474495787' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pg_num": 12}]: dispatch 2024-02-05T08:07:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:24 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4232727246' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:24 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:25.436 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.435+0000 7f31b9ffb640 1 -- 172.21.15.43:0/4232727246 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v770) v1 ==== 223+0+0 (secure 0 0 0) 0x7f31a4043020 con 0x7f31bc19b920 2024-02-05T08:07:25.436 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:07:25.438 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f31a806da30 msgr2=0x7f31a806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:25.438 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f31a806da30 0x7f31a806fef0 secure :-1 s=READY pgs=4060 cs=0 l=1 rev1=1 crypto rx=0x7f31b4005c40 tx=0x7f31b400a570 comp rx=0 tx=0).stop 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc19b920 msgr2=0x7f31bc19bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc19b920 0x7f31bc19bdd0 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f31a4005940 tx=0x7f31a40042f0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 shutdown_connections 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f31a806da30 0x7f31a806fef0 unknown :-1 s=CLOSED pgs=4060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f31bc19b920 0x7f31bc19bdd0 unknown :-1 s=CLOSED pgs=3146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f31bc1065d0 0x7f31bc197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 --2- 172.21.15.43:0/4232727246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31bc101500 0x7f31bc196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 >> 172.21.15.43:0/4232727246 conn(0x7f31bc0fcd60 msgr2=0x7f31bc0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 shutdown_connections 2024-02-05T08:07:25.439 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.438+0000 7f31c2d7a640 1 -- 172.21.15.43:0/4232727246 wait complete. 2024-02-05T08:07:25.450 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable e2a6ede4-caf1-4034-97c2-ddac1835f21e rados 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 -- 172.21.15.43:0/3752267201 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41054a0 msgr2=0x7f17e410d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/3752267201 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41054a0 0x7f17e410d070 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7f17dc009be0 tx=0x7f17dc02bef0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 -- 172.21.15.43:0/3752267201 shutdown_connections 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/3752267201 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41054a0 0x7f17e410d070 unknown :-1 s=CLOSED pgs=3147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/3752267201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e4100d80 0x7f17e4104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/3752267201 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17e41003d0 0x7f17e41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 -- 172.21.15.43:0/3752267201 >> 172.21.15.43:0/3752267201 conn(0x7f17e4077230 msgr2=0x7f17e4077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:25.542 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 -- 172.21.15.43:0/3752267201 shutdown_connections 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.539+0000 7f17eb5ef640 1 -- 172.21.15.43:0/3752267201 wait complete. 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 Processor -- start 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 -- start start 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41003d0 0x7f17e4101e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41054a0 0x7f17e41023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17e41029f0 0x7f17e41a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e41a58d0 con 0x7f17e41054a0 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e41a5a40 con 0x7f17e41029f0 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17eb5ef640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e41a5c00 con 0x7f17e41003d0 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41003d0 0x7f17e4101e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e8b8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41054a0 0x7f17e41023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41003d0 0x7f17e4101e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:48714/0 (socket says 172.21.15.43:48714) 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 -- 172.21.15.43:0/2280202348 learned_addr learned my addr 172.21.15.43:0/2280202348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e9b8c640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17e41029f0 0x7f17e41a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:25.543 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 -- 172.21.15.43:0/2280202348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17e41029f0 msgr2=0x7f17e41a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17e41029f0 0x7f17e41a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 -- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41054a0 msgr2=0x7f17e41023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41054a0 0x7f17e41023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e938b640 1 -- 172.21.15.43:0/2280202348 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d8009560 con 0x7f17e41003d0 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e8b8a640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41054a0 0x7f17e41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.540+0000 7f17e9b8c640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17e41029f0 0x7f17e41a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.541+0000 7f17e938b640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41003d0 0x7f17e4101e60 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e9f0 tx=0x7f17e000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.541+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0009720 con 0x7f17e41003d0 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.541+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc009840 con 0x7f17e41003d0 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.541+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f17e000cd80 con 0x7f17e41003d0 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.541+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0004ea0 con 0x7f17e41003d0 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.541+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e41a6210 con 0x7f17e41003d0 2024-02-05T08:07:25.544 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.542+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e4077d50 con 0x7f17e41003d0 2024-02-05T08:07:25.545 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.542+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f17e0005020 con 0x7f17e41003d0 2024-02-05T08:07:25.545 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.543+0000 7f17d67fc640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f17d006da30 0x7f17d006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:25.545 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.543+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(770..770 src has 257..770) v4 ==== 9054+0+0 (secure 0 0 0) 0x7f17e0014070 con 0x7f17e41003d0 2024-02-05T08:07:25.545 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.543+0000 7f17e8b8a640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f17d006da30 0x7f17d006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:25.545 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.543+0000 7f17e8b8a640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f17d006da30 0x7f17d006fef0 secure :-1 s=READY pgs=4061 cs=0 l=1 rev1=1 crypto rx=0x7f17e4103b00 tx=0x7f17d8009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:25.547 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.547+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17e00571b0 con 0x7f17e41003d0 2024-02-05T08:07:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:25.730+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"} v 0) v1 -- 0x7f17e404f070 con 0x7f17e41003d0 2024-02-05T08:07:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:07:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[29879]: osdmap e769: 8 total, 8 up, 8 in 2024-02-05T08:07:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4232727246' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:25.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:25.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:07:25.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[41444]: osdmap e769: 8 total, 8 up, 8 in 2024-02-05T08:07:25.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4232727246' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:25.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:25 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:07:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:25 smithi050 ceph-mon[33492]: osdmap e769: 8 total, 8 up, 8 in 2024-02-05T08:07:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:25 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4232727246' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:25.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:25 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:07:26.436 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:26.436+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]=0 enabled application 'rados' on pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' v771) v1 ==== 213+0+0 (secure 0 0 0) 0x7f17e005a870 con 0x7f17e41003d0 2024-02-05T08:07:26.549 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:26.548+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"} v 0) v1 -- 0x7f17e4100d80 con 0x7f17e41003d0 2024-02-05T08:07:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[29879]: pgmap v1681: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T08:07:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[29879]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:07:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[29879]: osdmap e770: 8 total, 8 up, 8 in 2024-02-05T08:07:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2280202348' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[41444]: pgmap v1681: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T08:07:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[41444]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:07:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[41444]: osdmap e770: 8 total, 8 up, 8 in 2024-02-05T08:07:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2280202348' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:26 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:26 smithi050 ceph-mon[33492]: pgmap v1681: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-05T08:07:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:26 smithi050 ceph-mon[33492]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:07:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:26 smithi050 ceph-mon[33492]: osdmap e770: 8 total, 8 up, 8 in 2024-02-05T08:07:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:26 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2280202348' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:26 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:27.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.447+0000 7f17d67fc640 1 -- 172.21.15.43:0/2280202348 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]=0 enabled application 'rados' on pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' v772) v1 ==== 213+0+0 (secure 0 0 0) 0x7f17e005aa50 con 0x7f17e41003d0 2024-02-05T08:07:27.448 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f17d006da30 msgr2=0x7f17d006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f17d006da30 0x7f17d006fef0 secure :-1 s=READY pgs=4061 cs=0 l=1 rev1=1 crypto rx=0x7f17e4103b00 tx=0x7f17d8009260 comp rx=0 tx=0).stop 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41003d0 msgr2=0x7f17e4101e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41003d0 0x7f17e4101e60 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f17e000e9f0 tx=0x7f17e000edf0 comp rx=0 tx=0).stop 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 shutdown_connections 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f17d006da30 0x7f17d006fef0 unknown :-1 s=CLOSED pgs=4061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f17e41029f0 0x7f17e41a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17e41054a0 0x7f17e41023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 --2- 172.21.15.43:0/2280202348 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f17e41003d0 0x7f17e4101e60 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 >> 172.21.15.43:0/2280202348 conn(0x7f17e4077230 msgr2=0x7f17e4069850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:27.451 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 shutdown_connections 2024-02-05T08:07:27.452 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:27.450+0000 7f17eb5ef640 1 -- 172.21.15.43:0/2280202348 wait complete. 2024-02-05T08:07:27.463 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-02-05T08:07:27.471 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.471 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj1 /etc/passwd 2024-02-05T08:07:27.509 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.509 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj2 /etc/passwd 2024-02-05T08:07:27.548 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.548 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj3 /etc/passwd 2024-02-05T08:07:27.584 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.584 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj4 /etc/passwd 2024-02-05T08:07:27.620 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.620 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj5 /etc/passwd 2024-02-05T08:07:27.654 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.655 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj6 /etc/passwd 2024-02-05T08:07:27.689 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.689 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj7 /etc/passwd 2024-02-05T08:07:27.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj8 /etc/passwd 2024-02-05T08:07:27.758 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.759 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj9 /etc/passwd 2024-02-05T08:07:27.793 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:07:27.793 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put obj10 /etc/passwd 2024-02-05T08:07:27.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]': finished 2024-02-05T08:07:27.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[41444]: osdmap e771: 8 total, 8 up, 8 in 2024-02-05T08:07:27.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2280202348' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:27.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:27.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[41444]: pgmap v1684: 176 pgs: 176 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]': finished 2024-02-05T08:07:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[29879]: osdmap e771: 8 total, 8 up, 8 in 2024-02-05T08:07:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2280202348' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:27.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:27 smithi043 ceph-mon[29879]: pgmap v1684: 176 pgs: 176 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]': finished 2024-02-05T08:07:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:27 smithi050 ceph-mon[33492]: osdmap e771: 8 total, 8 up, 8 in 2024-02-05T08:07:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:27 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2280202348' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:27 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]: dispatch 2024-02-05T08:07:27.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:27 smithi050 ceph-mon[33492]: pgmap v1684: 176 pgs: 176 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:27.828 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-02-05T08:07:28.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:28 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]': finished 2024-02-05T08:07:28.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:28 smithi050 ceph-mon[33492]: osdmap e772: 8 total, 8 up, 8 in 2024-02-05T08:07:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:28 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]': finished 2024-02-05T08:07:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:28 smithi043 ceph-mon[29879]: osdmap e772: 8 total, 8 up, 8 in 2024-02-05T08:07:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:28 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "app": "rados"}]': finished 2024-02-05T08:07:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:28 smithi043 ceph-mon[41444]: osdmap e772: 8 total, 8 up, 8 in 2024-02-05T08:07:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:07:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:07:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:29.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:29.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:29 smithi043 ceph-mon[29879]: pgmap v1686: 176 pgs: 176 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:07:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:29 smithi043 ceph-mon[41444]: pgmap v1686: 176 pgs: 176 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:07:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:29.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:29 smithi050 ceph-mon[33492]: pgmap v1686: 176 pgs: 176 active+clean; 582 KiB data, 853 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:07:32.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:07:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:07:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:32 smithi050 ceph-mon[33492]: pgmap v1687: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-02-05T08:07:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:32 smithi050 ceph-mon[33492]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:32.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:32 smithi050 ceph-mon[33492]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' is full (reached quota's max_objects: 10) 2024-02-05T08:07:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:32 smithi050 ceph-mon[33492]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:07:32.573 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:32 smithi050 ceph-mon[33492]: osdmap e773: 8 total, 8 up, 8 in 2024-02-05T08:07:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[29879]: pgmap v1687: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-02-05T08:07:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[29879]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[29879]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' is full (reached quota's max_objects: 10) 2024-02-05T08:07:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[29879]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:07:32.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[29879]: osdmap e773: 8 total, 8 up, 8 in 2024-02-05T08:07:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[41444]: pgmap v1687: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 20 KiB/s wr, 3 op/s 2024-02-05T08:07:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[41444]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:07:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[41444]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' is full (reached quota's max_objects: 10) 2024-02-05T08:07:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[41444]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:07:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:32 smithi043 ceph-mon[41444]: osdmap e773: 8 total, 8 up, 8 in 2024-02-05T08:07:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:33.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:34 smithi043 ceph-mon[29879]: pgmap v1689: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 17 KiB/s wr, 2 op/s 2024-02-05T08:07:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:34 smithi043 ceph-mon[41444]: pgmap v1689: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 17 KiB/s wr, 2 op/s 2024-02-05T08:07:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:34 smithi050 ceph-mon[33492]: pgmap v1689: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 17 KiB/s wr, 2 op/s 2024-02-05T08:07:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:36 smithi043 ceph-mon[29879]: pgmap v1690: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-05T08:07:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:36 smithi043 ceph-mon[41444]: pgmap v1690: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-05T08:07:36.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:36 smithi050 ceph-mon[33492]: pgmap v1690: 176 pgs: 176 active+clean; 695 KiB data, 848 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 15 KiB/s wr, 1 op/s 2024-02-05T08:07:38.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:38 smithi043 ceph-mon[29879]: pgmap v1691: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-02-05T08:07:38.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:38 smithi043 ceph-mon[41444]: pgmap v1691: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-02-05T08:07:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:38 smithi050 ceph-mon[33492]: pgmap v1691: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-02-05T08:07:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:07:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:07:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:40 smithi043 ceph-mon[29879]: pgmap v1692: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:07:40.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:40 smithi043 ceph-mon[41444]: pgmap v1692: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:07:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:40 smithi050 ceph-mon[33492]: pgmap v1692: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:07:42.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:42 smithi050 ceph-mon[33492]: pgmap v1693: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:42.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:07:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:07:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:42 smithi043 ceph-mon[29879]: pgmap v1693: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:42.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:42 smithi043 ceph-mon[41444]: pgmap v1693: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:44 smithi043 ceph-mon[29879]: pgmap v1694: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-05T08:07:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:44 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:07:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:44 smithi043 ceph-mon[41444]: pgmap v1694: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-05T08:07:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:44 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:07:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:44 smithi050 ceph-mon[33492]: pgmap v1694: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-05T08:07:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:44 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:07:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:46.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:46 smithi043 ceph-mon[29879]: pgmap v1695: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:46 smithi043 ceph-mon[41444]: pgmap v1695: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:46 smithi050 ceph-mon[33492]: pgmap v1695: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:48.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:48 smithi043 ceph-mon[29879]: pgmap v1696: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:48.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:48 smithi043 ceph-mon[41444]: pgmap v1696: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:48.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:48 smithi050 ceph-mon[33492]: pgmap v1696: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:07:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:07:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:50 smithi043 ceph-mon[29879]: pgmap v1697: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:50.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:50 smithi043 ceph-mon[41444]: pgmap v1697: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:50.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:50 smithi050 ceph-mon[33492]: pgmap v1697: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:52.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:52 smithi050 ceph-mon[33492]: pgmap v1698: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:52.573 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:07:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:07:52.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:52 smithi043 ceph-mon[29879]: pgmap v1698: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:52.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:52 smithi043 ceph-mon[41444]: pgmap v1698: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:53.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:53.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:53.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:07:54.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:54 smithi043 ceph-mon[29879]: pgmap v1699: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:54.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:54 smithi043 ceph-mon[41444]: pgmap v1699: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:54.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:54 smithi050 ceph-mon[33492]: pgmap v1699: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:56.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:56 smithi043 ceph-mon[29879]: pgmap v1700: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:56.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:56 smithi043 ceph-mon[41444]: pgmap v1700: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:56.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:56 smithi050 ceph-mon[33492]: pgmap v1700: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:07:57.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=158496 2024-02-05T08:07:57.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota e2a6ede4-caf1-4034-97c2-ddac1835f21e max_objects 100 2024-02-05T08:07:57.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put onemore /etc/passwd 2024-02-05T08:07:57.931 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 -- 172.21.15.43:0/248834065 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102e30 msgr2=0x7f405010b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 --2- 172.21.15.43:0/248834065 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102e30 0x7f405010b310 secure :-1 s=READY pgs=3169 cs=0 l=1 rev1=1 crypto rx=0x7f4044009900 tx=0x7f404402bd60 comp rx=0 tx=0).stop 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 -- 172.21.15.43:0/248834065 shutdown_connections 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 --2- 172.21.15.43:0/248834065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f405010b850 0x7f405010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 --2- 172.21.15.43:0/248834065 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102e30 0x7f405010b310 unknown :-1 s=CLOSED pgs=3169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 --2- 172.21.15.43:0/248834065 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f4050102510 0x7f40501028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 -- 172.21.15.43:0/248834065 >> 172.21.15.43:0/248834065 conn(0x7f4050077230 msgr2=0x7f4050077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 -- 172.21.15.43:0/248834065 shutdown_connections 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.926+0000 7f4056627640 1 -- 172.21.15.43:0/248834065 wait complete. 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 Processor -- start 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 -- start start 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102510 0x7f4050196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:57.932 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4050102e30 0x7f4050197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f405010b850 0x7f405019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019bec0 con 0x7f4050102e30 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405019c030 con 0x7f405010b850 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4056627640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405019c1f0 con 0x7f4050102510 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f404ffff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102510 0x7f4050196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f404f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4050102e30 0x7f4050197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f404f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4050102e30 0x7f4050197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44182/0 (socket says 172.21.15.43:44182) 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f404f7fe640 1 -- 172.21.15.43:0/2269840699 learned_addr learned my addr 172.21.15.43:0/2269840699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.927+0000 7f4054bc4640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f405010b850 0x7f405019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404f7fe640 1 -- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102510 msgr2=0x7f4050196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404f7fe640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102510 0x7f4050196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404f7fe640 1 -- 172.21.15.43:0/2269840699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f405010b850 msgr2=0x7f405019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:57.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404f7fe640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f405010b850 0x7f405019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404f7fe640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4048009840 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404f7fe640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4050102e30 0x7f4050197330 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f4044006010 tx=0x7f40440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40440044e0 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4044003c80 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4044009560 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40440053d0 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.928+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40501a3b70 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.929+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4050077d50 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.930+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f4044005530 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.930+0000 7f404d7fa640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f403806da30 0x7f403806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.930+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(773..773 src has 257..773) v4 ==== 9067+0+0 (secure 0 0 0) 0x7f40440b0890 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.930+0000 7f404ffff640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f403806da30 0x7f403806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.931+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7f40380794c0 con 0x7f4050102e30 2024-02-05T08:07:57.934 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.931+0000 7f404ffff640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f403806da30 0x7f403806fef0 secure :-1 s=READY pgs=4073 cs=0 l=1 rev1=1 crypto rx=0x7f4040005d00 tx=0x7f404000b410 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:07:57.936 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:57.935+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4044079640 con 0x7f4050102e30 2024-02-05T08:07:58.118 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:58.118+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f405004f070 con 0x7f4050102e30 2024-02-05T08:07:58.427 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:58.426+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v774) v1 ==== 225+0+0 (secure 0 0 0) 0x7f4044043020 con 0x7f4050102e30 2024-02-05T08:07:58.427 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:58.426+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(774..774 src has 257..774) v4 ==== 628+0+0 (secure 0 0 0) 0x7f4044004d90 con 0x7f4050102e30 2024-02-05T08:07:58.427 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:58.426+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=775}) v3 -- 0x7f403807a3a0 con 0x7f4050102e30 2024-02-05T08:07:58.540 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:58.540+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f40501060e0 con 0x7f4050102e30 2024-02-05T08:07:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:58 smithi043 ceph-mon[29879]: pgmap v1701: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:58 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:07:58.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:58 smithi043 ceph-mon[41444]: pgmap v1701: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:58.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:58 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:07:58.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:58 smithi050 ceph-mon[33492]: pgmap v1701: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:07:58.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:58 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:07:59.440 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.439+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v775) v1 ==== 225+0+0 (secure 0 0 0) 0x7f4044043070 con 0x7f4050102e30 2024-02-05T08:07:59.440 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 100 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:07:59.440 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.439+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 <== mon.0 v2:172.21.15.43:3300/0 10 ==== osd_map(775..775 src has 257..775) v4 ==== 628+0+0 (secure 0 0 0) 0x7f404407daa0 con 0x7f4050102e30 2024-02-05T08:07:59.440 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.439+0000 7f404d7fa640 1 -- 172.21.15.43:0/2269840699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=776}) v3 -- 0x7f403807a3e0 con 0x7f4050102e30 2024-02-05T08:07:59.446 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.446+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f403806da30 msgr2=0x7f403806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:59.446 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.446+0000 7f4056627640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f403806da30 0x7f403806fef0 secure :-1 s=READY pgs=4073 cs=0 l=1 rev1=1 crypto rx=0x7f4040005d00 tx=0x7f404000b410 comp rx=0 tx=0).stop 2024-02-05T08:07:59.446 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.446+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4050102e30 msgr2=0x7f4050197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:07:59.447 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.446+0000 7f4056627640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4050102e30 0x7f4050197330 secure :-1 s=READY pgs=3006 cs=0 l=1 rev1=1 crypto rx=0x7f4044006010 tx=0x7f40440042f0 comp rx=0 tx=0).stop 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.447+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 shutdown_connections 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.447+0000 7f4056627640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f403806da30 0x7f403806fef0 unknown :-1 s=CLOSED pgs=4073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.447+0000 7f4056627640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f405010b850 0x7f405019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.447+0000 7f4056627640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4050102e30 0x7f4050197330 unknown :-1 s=CLOSED pgs=3006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.447+0000 7f4056627640 1 --2- 172.21.15.43:0/2269840699 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f4050102510 0x7f4050196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.447+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 >> 172.21.15.43:0/2269840699 conn(0x7f4050077230 msgr2=0x7f40501032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.447+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 shutdown_connections 2024-02-05T08:07:59.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:07:59.448+0000 7f4056627640 1 -- 172.21.15.43:0/2269840699 wait complete. 2024-02-05T08:07:59.461 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 158496 2024-02-05T08:07:59.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:07:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[29879]: osdmap e774: 8 total, 8 up, 8 in 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[29879]: pgmap v1703: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[41444]: osdmap e774: 8 total, 8 up, 8 in 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:07:59.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:07:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:59.564 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:07:59 smithi043 ceph-mon[41444]: pgmap v1703: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:07:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:07:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:59 smithi050 ceph-mon[33492]: osdmap e774: 8 total, 8 up, 8 in 2024-02-05T08:07:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:59 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]: dispatch 2024-02-05T08:07:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:07:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:07:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:07:59 smithi050 ceph-mon[33492]: pgmap v1703: 176 pgs: 176 active+clean; 695 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:08:00.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:00 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:08:00.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:00 smithi043 ceph-mon[29879]: osdmap e775: 8 total, 8 up, 8 in 2024-02-05T08:08:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:00 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:08:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:00 smithi043 ceph-mon[41444]: osdmap e775: 8 total, 8 up, 8 in 2024-02-05T08:08:00.818 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:00 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2269840699' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "100"}]': finished 2024-02-05T08:08:00.818 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:00 smithi050 ceph-mon[33492]: osdmap e775: 8 total, 8 up, 8 in 2024-02-05T08:08:01.406 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-02-05T08:08:01.407 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-02-05T08:08:01.407 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put twomore /etc/passwd 2024-02-05T08:08:01.442 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota e2a6ede4-caf1-4034-97c2-ddac1835f21e max_bytes 100 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 -- 172.21.15.43:0/588538870 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 msgr2=0x7fdcfc069ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 --2- 172.21.15.43:0/588538870 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc069ee0 secure :-1 s=READY pgs=3170 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009900 tx=0x7fdcec02bc30 comp rx=0 tx=0).stop 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 -- 172.21.15.43:0/588538870 shutdown_connections 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 --2- 172.21.15.43:0/588538870 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcfc106570 0x7fdcfc10e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 --2- 172.21.15.43:0/588538870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc06a4b0 0x7fdcfc105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 --2- 172.21.15.43:0/588538870 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc069ee0 unknown :-1 s=CLOSED pgs=3170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 -- 172.21.15.43:0/588538870 >> 172.21.15.43:0/588538870 conn(0x7fdcfc076f90 msgr2=0x7fdcfc0773a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 -- 172.21.15.43:0/588538870 shutdown_connections 2024-02-05T08:08:01.527 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.526+0000 7fdd010ae640 1 -- 172.21.15.43:0/588538870 wait complete. 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 Processor -- start 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 -- start start 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc106570 0x7fdcfc197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcfc19b8b0 0x7fdcfc19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc19c2a0 con 0x7fdcfc106570 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc19c410 con 0x7fdcfc19b8b0 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdd010ae640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcfc19c580 con 0x7fdcfc069b00 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdcfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc106570 0x7fdcfc197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdcfad9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdcfa59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc106570 0x7fdcfc197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48784/0 (socket says 172.21.15.43:48784) 2024-02-05T08:08:01.530 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdcfa59c640 1 -- 172.21.15.43:0/4250815314 learned_addr learned my addr 172.21.15.43:0/4250815314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdcfad9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:40238/0 (socket says 172.21.15.43:40238) 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.527+0000 7fdcfb59e640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcfc19b8b0 0x7fdcfc19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfa59c640 1 -- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 msgr2=0x7fdcfc196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfa59c640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfa59c640 1 -- 172.21.15.43:0/4250815314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcfc19b8b0 msgr2=0x7fdcfc19bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfa59c640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcfc19b8b0 0x7fdcfc19bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfa59c640 1 -- 172.21.15.43:0/4250815314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4009490 con 0x7fdcfc106570 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfad9d640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfb59e640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcfc19b8b0 0x7fdcfc19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcfa59c640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc106570 0x7fdcfc197320 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7fdce400e9a0 tx=0x7fdce400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce400cd80 con 0x7fdcfc106570 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcec009560 con 0x7fdcfc106570 2024-02-05T08:08:01.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdce40044a0 con 0x7fdcfc106570 2024-02-05T08:08:01.532 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce4010430 con 0x7fdcfc106570 2024-02-05T08:08:01.532 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.528+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcfc1a3ed0 con 0x7fdcfc106570 2024-02-05T08:08:01.532 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.529+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcfc077ab0 con 0x7fdcfc106570 2024-02-05T08:08:01.532 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.530+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fdce4003f60 con 0x7fdcfc106570 2024-02-05T08:08:01.532 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.530+0000 7fdcf3fff640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdce806da30 0x7fdce806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:01.532 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.530+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(776..776 src has 257..776) v4 ==== 9067+0+0 (secure 0 0 0) 0x7fdce4014070 con 0x7fdcfc106570 2024-02-05T08:08:01.533 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.530+0000 7fdcfad9d640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdce806da30 0x7fdce806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:01.533 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.530+0000 7fdcfad9d640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdce806da30 0x7fdce806fef0 secure :-1 s=READY pgs=4075 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009230 tx=0x7fdcec005fd0 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:01.535 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.535+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdce4057180 con 0x7fdcfc106570 2024-02-05T08:08:01.719 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:01.719+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7fdcfc04f070 con 0x7fdcfc106570 2024-02-05T08:08:01.772 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[29879]: pgmap v1705: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:01.773 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[29879]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:08:01.773 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[29879]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:08:01.773 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[29879]: osdmap e776: 8 total, 8 up, 8 in 2024-02-05T08:08:01.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[41444]: pgmap v1705: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:01.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[41444]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:08:01.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[41444]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:08:01.773 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:01 smithi043 ceph-mon[41444]: osdmap e776: 8 total, 8 up, 8 in 2024-02-05T08:08:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:01 smithi050 ceph-mon[33492]: pgmap v1705: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:01 smithi050 ceph-mon[33492]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:08:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:01 smithi050 ceph-mon[33492]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:08:01.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:01 smithi050 ceph-mon[33492]: osdmap e776: 8 total, 8 up, 8 in 2024-02-05T08:08:02.450 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:02.449+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v777) v1 ==== 221+0+0 (secure 0 0 0) 0x7fdce405a840 con 0x7fdcfc106570 2024-02-05T08:08:02.565 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:02.564+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7fdcfc06a4b0 con 0x7fdcfc106570 2024-02-05T08:08:02.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:08:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:08:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:02 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:08:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:02 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:02 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:02.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:02 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:08:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:02.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:08:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:02.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:02 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:03.454 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.454+0000 7fdcf3fff640 1 -- 172.21.15.43:0/4250815314 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v778) v1 ==== 221+0+0 (secure 0 0 0) 0x7fdce405aa20 con 0x7fdcfc106570 2024-02-05T08:08:03.454 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 100 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:08:03.459 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.458+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdce806da30 msgr2=0x7fdce806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:03.459 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.458+0000 7fdd010ae640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdce806da30 0x7fdce806fef0 secure :-1 s=READY pgs=4075 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009230 tx=0x7fdcec005fd0 comp rx=0 tx=0).stop 2024-02-05T08:08:03.459 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.458+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc106570 msgr2=0x7fdcfc197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:03.459 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.458+0000 7fdd010ae640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc106570 0x7fdcfc197320 secure :-1 s=READY pgs=3010 cs=0 l=1 rev1=1 crypto rx=0x7fdce400e9a0 tx=0x7fdce400ee60 comp rx=0 tx=0).stop 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 shutdown_connections 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fdce806da30 0x7fdce806fef0 unknown :-1 s=CLOSED pgs=4075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fdcfc19b8b0 0x7fdcfc19bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdcfc106570 0x7fdcfc197320 unknown :-1 s=CLOSED pgs=3010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 --2- 172.21.15.43:0/4250815314 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fdcfc069b00 0x7fdcfc196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 >> 172.21.15.43:0/4250815314 conn(0x7fdcfc076f90 msgr2=0x7fdcfc0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 shutdown_connections 2024-02-05T08:08:03.460 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:03.459+0000 7fdd010ae640 1 -- 172.21.15.43:0/4250815314 wait complete. 2024-02-05T08:08:03.471 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-02-05T08:08:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:08:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[29879]: osdmap e777: 8 total, 8 up, 8 in 2024-02-05T08:08:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:08:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[29879]: pgmap v1708: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:08:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:08:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[41444]: osdmap e777: 8 total, 8 up, 8 in 2024-02-05T08:08:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:08:03.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:03 smithi043 ceph-mon[41444]: pgmap v1708: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:08:03.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:08:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:03 smithi050 ceph-mon[33492]: osdmap e777: 8 total, 8 up, 8 in 2024-02-05T08:08:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:03 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-05T08:08:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:03 smithi050 ceph-mon[33492]: pgmap v1708: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:08:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:08:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[29879]: osdmap e778: 8 total, 8 up, 8 in 2024-02-05T08:08:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[41444]: osdmap e778: 8 total, 8 up, 8 in 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:08:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:04 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:04 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4250815314' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "100"}]': finished 2024-02-05T08:08:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:04 smithi050 ceph-mon[33492]: osdmap e778: 8 total, 8 up, 8 in 2024-02-05T08:08:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:08:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:08:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:04 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:05 smithi043 ceph-mon[29879]: pgmap v1710: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 871 B/s rd, 0 op/s 2024-02-05T08:08:05.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:05 smithi043 ceph-mon[41444]: pgmap v1710: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 871 B/s rd, 0 op/s 2024-02-05T08:08:05.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:05 smithi050 ceph-mon[33492]: pgmap v1710: 176 pgs: 176 active+clean; 695 KiB data, 842 MiB used, 714 GiB / 715 GiB avail; 871 B/s rd, 0 op/s 2024-02-05T08:08:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:06 smithi043 ceph-mon[29879]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' is full (reached quota's max_bytes: 100 B) 2024-02-05T08:08:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:06 smithi043 ceph-mon[29879]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:08:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:06 smithi043 ceph-mon[29879]: osdmap e779: 8 total, 8 up, 8 in 2024-02-05T08:08:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:06 smithi043 ceph-mon[41444]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' is full (reached quota's max_bytes: 100 B) 2024-02-05T08:08:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:06 smithi043 ceph-mon[41444]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:08:06.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:06 smithi043 ceph-mon[41444]: osdmap e779: 8 total, 8 up, 8 in 2024-02-05T08:08:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:06 smithi050 ceph-mon[33492]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' is full (reached quota's max_bytes: 100 B) 2024-02-05T08:08:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:06 smithi050 ceph-mon[33492]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:08:06.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:06 smithi050 ceph-mon[33492]: osdmap e779: 8 total, 8 up, 8 in 2024-02-05T08:08:07.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:07 smithi043 ceph-mon[29879]: pgmap v1712: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 864 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T08:08:07.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:07 smithi043 ceph-mon[41444]: pgmap v1712: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 864 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T08:08:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:07 smithi050 ceph-mon[33492]: pgmap v1712: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 864 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T08:08:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:08:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:08:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:10.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:10 smithi043 ceph-mon[41444]: pgmap v1713: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 745 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-05T08:08:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:10 smithi043 ceph-mon[29879]: pgmap v1713: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 745 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-05T08:08:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:10 smithi050 ceph-mon[33492]: pgmap v1713: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 745 B/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-05T08:08:12.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:12 smithi050 ceph-mon[33492]: pgmap v1714: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:08:12.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:08:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:08:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:12 smithi043 ceph-mon[29879]: pgmap v1714: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:08:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:12 smithi043 ceph-mon[41444]: pgmap v1714: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:08:13.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:13.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:14.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:14 smithi043 ceph-mon[41444]: pgmap v1715: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:08:14.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:14 smithi043 ceph-mon[29879]: pgmap v1715: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:08:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:14 smithi050 ceph-mon[33492]: pgmap v1715: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:08:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:16 smithi043 ceph-mon[29879]: pgmap v1716: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:08:16.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:16 smithi043 ceph-mon[41444]: pgmap v1716: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:08:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:16 smithi050 ceph-mon[33492]: pgmap v1716: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-05T08:08:18.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:18 smithi043 ceph-mon[29879]: pgmap v1717: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-05T08:08:18.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:18 smithi043 ceph-mon[41444]: pgmap v1717: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-05T08:08:18.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:18 smithi050 ceph-mon[33492]: pgmap v1717: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 937 B/s rd, 0 op/s 2024-02-05T08:08:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:08:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:08:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:20.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:20 smithi043 ceph-mon[29879]: pgmap v1718: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:20.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:20 smithi043 ceph-mon[41444]: pgmap v1718: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:20.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:20 smithi050 ceph-mon[33492]: pgmap v1718: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:22.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:22 smithi050 ceph-mon[33492]: pgmap v1719: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:22.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:08:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:08:22.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:22 smithi043 ceph-mon[29879]: pgmap v1719: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:22.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:22 smithi043 ceph-mon[41444]: pgmap v1719: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:23.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:24.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:24 smithi043 ceph-mon[29879]: pgmap v1720: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:24.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:24 smithi043 ceph-mon[41444]: pgmap v1720: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:24 smithi050 ceph-mon[33492]: pgmap v1720: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:26 smithi043 ceph-mon[29879]: pgmap v1721: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:26.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:26 smithi043 ceph-mon[41444]: pgmap v1721: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:26.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:26 smithi050 ceph-mon[33492]: pgmap v1721: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:28.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:28 smithi043 ceph-mon[29879]: pgmap v1722: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:28.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:28 smithi043 ceph-mon[41444]: pgmap v1722: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:28.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:28 smithi050 ceph-mon[33492]: pgmap v1722: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:29.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:29 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:08:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:08:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:29.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:29 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:29.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:29 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:30.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:30 smithi043 ceph-mon[29879]: pgmap v1723: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:30.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:30 smithi043 ceph-mon[41444]: pgmap v1723: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:30.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:30 smithi050 ceph-mon[33492]: pgmap v1723: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:32 smithi043 ceph-mon[29879]: pgmap v1724: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:32.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:32 smithi043 ceph-mon[41444]: pgmap v1724: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:32.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:32 smithi050 ceph-mon[33492]: pgmap v1724: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:32.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:08:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:08:33.472 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=159209 2024-02-05T08:08:33.473 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota e2a6ede4-caf1-4034-97c2-ddac1835f21e max_bytes 0 2024-02-05T08:08:33.473 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put two /etc/passwd 2024-02-05T08:08:33.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 -- 172.21.15.43:0/3468398478 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe828100d50 msgr2=0x7fe828104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/3468398478 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe828100d50 0x7fe828104d90 secure :-1 s=READY pgs=3173 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009860 tx=0x7fe81c02bb70 comp rx=0 tx=0).stop 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 -- 172.21.15.43:0/3468398478 shutdown_connections 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/3468398478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe828105470 0x7fe82810d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/3468398478 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe828100d50 0x7fe828104d90 unknown :-1 s=CLOSED pgs=3173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/3468398478 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe8281003a0 0x7fe828100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 -- 172.21.15.43:0/3468398478 >> 172.21.15.43:0/3468398478 conn(0x7fe8280764b0 msgr2=0x7fe8280768c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 -- 172.21.15.43:0/3468398478 shutdown_connections 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 -- 172.21.15.43:0/3468398478 wait complete. 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.568+0000 7fe82d9e2640 1 Processor -- start 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe82d9e2640 1 -- start start 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe82d9e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8281003a0 0x7fe828197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:33.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe82d9e2640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe828105470 0x7fe828198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe82d9e2640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8281a0d60 0x7fe8281a3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe82d9e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8281a3690 con 0x7fe8281003a0 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe82d9e2640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8281a37d0 con 0x7fe828105470 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe82d9e2640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8281a3960 con 0x7fe8281a0d60 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8281003a0 0x7fe828197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8281003a0 0x7fe828197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41372/0 (socket says 172.21.15.43:41372) 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 -- 172.21.15.43:0/2439268767 learned_addr learned my addr 172.21.15.43:0/2439268767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe8277fe640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8281a0d60 0x7fe8281a3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe8267fc640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe828105470 0x7fe828198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 -- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8281a0d60 msgr2=0x7fe8281a3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8281a0d60 0x7fe8281a3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 -- 172.21.15.43:0/2439268767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe828105470 msgr2=0x7fe828198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:33.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe828105470 0x7fe828198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820009900 con 0x7fe8281003a0 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe8277fe640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8281a0d60 0x7fe8281a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.569+0000 7fe826ffd640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8281003a0 0x7fe828197ed0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7fe81800ea20 tx=0x7fe81800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.570+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818009720 con 0x7fe8281003a0 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.570+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe81800cd80 con 0x7fe8281003a0 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.570+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c009490 con 0x7fe8281003a0 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.570+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe818004e10 con 0x7fe8281003a0 2024-02-05T08:08:33.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.570+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8281a3f40 con 0x7fe8281003a0 2024-02-05T08:08:33.574 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.571+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe828076fd0 con 0x7fe8281003a0 2024-02-05T08:08:33.574 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.571+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fe818004f90 con 0x7fe8281003a0 2024-02-05T08:08:33.574 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.572+0000 7fe82c9e0640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe81006da30 0x7fe81006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:33.574 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.572+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(779..779 src has 257..779) v4 ==== 9067+0+0 (secure 0 0 0) 0x7fe818014070 con 0x7fe8281003a0 2024-02-05T08:08:33.574 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.572+0000 7fe8267fc640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe81006da30 0x7fe81006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:33.574 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.572+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7fe8100794b0 con 0x7fe8281003a0 2024-02-05T08:08:33.575 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.572+0000 7fe8267fc640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe81006da30 0x7fe81006fef0 secure :-1 s=READY pgs=4077 cs=0 l=1 rev1=1 crypto rx=0x7fe81c02c180 tx=0x7fe81c0058b0 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:33.577 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.577+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8180570d0 con 0x7fe8281003a0 2024-02-05T08:08:33.760 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:33.759+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe82804f070 con 0x7fe8281003a0 2024-02-05T08:08:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:33.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:34.442 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:34.442+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v780) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe81805a790 con 0x7fe8281003a0 2024-02-05T08:08:34.443 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:34.442+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(780..780 src has 257..780) v4 ==== 628+0+0 (secure 0 0 0) 0x7fe81805b530 con 0x7fe8281003a0 2024-02-05T08:08:34.443 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:34.442+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=781}) v3 -- 0x7fe81007a3d0 con 0x7fe8281003a0 2024-02-05T08:08:34.559 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:34.559+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe828100d50 con 0x7fe8281003a0 2024-02-05T08:08:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:34 smithi043 ceph-mon[29879]: pgmap v1725: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:34 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:08:34.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:34 smithi043 ceph-mon[41444]: pgmap v1725: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:34.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:34 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:08:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:34 smithi050 ceph-mon[33492]: pgmap v1725: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:34.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:34 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:08:35.456 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.455+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v781) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe81805a970 con 0x7fe8281003a0 2024-02-05T08:08:35.456 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:08:35.456 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.455+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 <== mon.0 v2:172.21.15.43:3300/0 10 ==== osd_map(781..781 src has 257..781) v4 ==== 628+0+0 (secure 0 0 0) 0x7fe81808e150 con 0x7fe8281003a0 2024-02-05T08:08:35.456 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.456+0000 7fe82c9e0640 1 -- 172.21.15.43:0/2439268767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=782}) v3 -- 0x7fe81007a410 con 0x7fe8281003a0 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe81006da30 msgr2=0x7fe81006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe81006da30 0x7fe81006fef0 secure :-1 s=READY pgs=4077 cs=0 l=1 rev1=1 crypto rx=0x7fe81c02c180 tx=0x7fe81c0058b0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8281003a0 msgr2=0x7fe828197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8281003a0 0x7fe828197ed0 secure :-1 s=READY pgs=3012 cs=0 l=1 rev1=1 crypto rx=0x7fe81800ea20 tx=0x7fe81800ee20 comp rx=0 tx=0).stop 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 shutdown_connections 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fe81006da30 0x7fe81006fef0 unknown :-1 s=CLOSED pgs=4077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fe8281a0d60 0x7fe8281a3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fe828105470 0x7fe828198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.461 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 --2- 172.21.15.43:0/2439268767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8281003a0 0x7fe828197ed0 unknown :-1 s=CLOSED pgs=3012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.462 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 >> 172.21.15.43:0/2439268767 conn(0x7fe8280764b0 msgr2=0x7fe8280fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:35.462 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.460+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 shutdown_connections 2024-02-05T08:08:35.462 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.461+0000 7fe82d9e2640 1 -- 172.21.15.43:0/2439268767 wait complete. 2024-02-05T08:08:35.474 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota e2a6ede4-caf1-4034-97c2-ddac1835f21e max_objects 0 2024-02-05T08:08:35.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 -- 172.21.15.43:0/3311204839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81003d0 msgr2=0x7f13e81007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:35.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 --2- 172.21.15.43:0/3311204839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81003d0 0x7f13e81007b0 secure :-1 s=READY pgs=3013 cs=0 l=1 rev1=1 crypto rx=0x7f13d8009890 tx=0x7f13d802bb70 comp rx=0 tx=0).stop 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 -- 172.21.15.43:0/3311204839 shutdown_connections 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 --2- 172.21.15.43:0/3311204839 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e81054a0 0x7f13e810d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 --2- 172.21.15.43:0/3311204839 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e8100d80 0x7f13e8104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 --2- 172.21.15.43:0/3311204839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e81003d0 0x7f13e81007b0 unknown :-1 s=CLOSED pgs=3013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 -- 172.21.15.43:0/3311204839 >> 172.21.15.43:0/3311204839 conn(0x7f13e8077230 msgr2=0x7f13e8077640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 -- 172.21.15.43:0/3311204839 shutdown_connections 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 -- 172.21.15.43:0/3311204839 wait complete. 2024-02-05T08:08:35.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.563+0000 7f13ed165640 1 Processor -- start 2024-02-05T08:08:35.566 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13ed165640 1 -- start start 2024-02-05T08:08:35.566 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13ed165640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e81003d0 0x7f13e80fdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:35.566 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13ed165640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e81054a0 0x7f13e80fe510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13ed165640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80febb0 0x7f13e81a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13ed165640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e81a5830 con 0x7f13e80febb0 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13ed165640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e81a59a0 con 0x7f13e81054a0 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13ed165640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13e81a5b60 con 0x7f13e81003d0 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80febb0 0x7f13e81a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e81003d0 0x7f13e80fdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e81003d0 0x7f13e80fdfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:44380/0 (socket says 172.21.15.43:44380) 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80febb0 0x7f13e81a50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41398/0 (socket says 172.21.15.43:41398) 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 -- 172.21.15.43:0/4159913924 learned_addr learned my addr 172.21.15.43:0/4159913924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e659c640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e81054a0 0x7f13e80fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 -- 172.21.15.43:0/4159913924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e81054a0 msgr2=0x7f13e80fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:35.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e81054a0 0x7f13e80fe510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 -- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80febb0 msgr2=0x7f13e81a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80febb0 0x7f13e81a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e6d9d640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d0009560 con 0x7f13e81003d0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e659c640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e81054a0 0x7f13e80fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.564+0000 7f13e759e640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80febb0 0x7f13e81a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.565+0000 7f13e6d9d640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e81003d0 0x7f13e80fdfd0 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d8004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.565+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d802c510 con 0x7f13e81003d0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.565+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f13d802caf0 con 0x7f13e81003d0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.565+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8009490 con 0x7f13e81003d0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.565+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d803d3f0 con 0x7f13e81003d0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.565+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e81a6100 con 0x7f13e81003d0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.566+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e8077d50 con 0x7f13e81003d0 2024-02-05T08:08:35.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.566+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f13d803b040 con 0x7f13e81003d0 2024-02-05T08:08:35.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.567+0000 7f13dffff640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f13d406da30 0x7f13d406fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:35.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.567+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(781..781 src has 257..781) v4 ==== 9067+0+0 (secure 0 0 0) 0x7f13d80af9d0 con 0x7f13e81003d0 2024-02-05T08:08:35.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.567+0000 7f13e659c640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f13d406da30 0x7f13d406fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:35.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.567+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=782}) v3 -- 0x7f13d40794c0 con 0x7f13e81003d0 2024-02-05T08:08:35.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.567+0000 7f13e659c640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f13d406da30 0x7f13d406fef0 secure :-1 s=READY pgs=4078 cs=0 l=1 rev1=1 crypto rx=0x7f13e80ffc70 tx=0x7f13d0009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:35.571 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.571+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f13d8078830 con 0x7f13e81003d0 2024-02-05T08:08:35.755 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:35.755+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f13e804f070 con 0x7f13e81003d0 2024-02-05T08:08:35.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:08:35.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[29879]: osdmap e780: 8 total, 8 up, 8 in 2024-02-05T08:08:35.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:08:35.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[29879]: pgmap v1727: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:08:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:08:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[41444]: osdmap e780: 8 total, 8 up, 8 in 2024-02-05T08:08:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:08:35.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:35 smithi043 ceph-mon[41444]: pgmap v1727: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:08:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:08:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:35 smithi050 ceph-mon[33492]: osdmap e780: 8 total, 8 up, 8 in 2024-02-05T08:08:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:35 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:08:35.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:35 smithi050 ceph-mon[33492]: pgmap v1727: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:08:36.408 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:36.408+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(782..782 src has 257..782) v4 ==== 628+0+0 (secure 0 0 0) 0x7f13d80035f0 con 0x7f13e81003d0 2024-02-05T08:08:36.409 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:36.408+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=783}) v3 -- 0x7f13d4079ff0 con 0x7f13e81003d0 2024-02-05T08:08:36.416 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:36.411+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v782) v1 ==== 221+0+0 (secure 0 0 0) 0x7f13d8039020 con 0x7f13e81003d0 2024-02-05T08:08:36.531 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:36.531+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f13e8100d80 con 0x7f13e81003d0 2024-02-05T08:08:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:08:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: osdmap e781: 8 total, 8 up, 8 in 2024-02-05T08:08:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4159913924' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:08:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[29879]: osdmap e782: 8 total, 8 up, 8 in 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: osdmap e781: 8 total, 8 up, 8 in 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4159913924' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:08:36.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:36 smithi043 ceph-mon[41444]: osdmap e782: 8 total, 8 up, 8 in 2024-02-05T08:08:36.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2439268767' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:08:36.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: osdmap e781: 8 total, 8 up, 8 in 2024-02-05T08:08:36.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4159913924' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:36.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:36.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' no longer out of quota; removing NO_QUOTA flag 2024-02-05T08:08:36.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:08:36.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:08:36.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:36 smithi050 ceph-mon[33492]: osdmap e782: 8 total, 8 up, 8 in 2024-02-05T08:08:37.463 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.462+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 9 ==== osd_map(783..783 src has 257..783) v4 ==== 628+0+0 (secure 0 0 0) 0x7f13d8080630 con 0x7f13e81003d0 2024-02-05T08:08:37.466 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.466+0000 7f13dffff640 1 -- 172.21.15.43:0/4159913924 <== mon.2 v2:172.21.15.43:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v783) v1 ==== 221+0+0 (secure 0 0 0) 0x7f13d8041070 con 0x7f13e81003d0 2024-02-05T08:08:37.467 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:08:37.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f13d406da30 msgr2=0x7f13d406fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:37.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f13d406da30 0x7f13d406fef0 secure :-1 s=READY pgs=4078 cs=0 l=1 rev1=1 crypto rx=0x7f13e80ffc70 tx=0x7f13d0009260 comp rx=0 tx=0).stop 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e81003d0 msgr2=0x7f13e80fdfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e81003d0 0x7f13e80fdfd0 secure :-1 s=READY pgs=3174 cs=0 l=1 rev1=1 crypto rx=0x7f13d8000c00 tx=0x7f13d8004040 comp rx=0 tx=0).stop 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 shutdown_connections 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f13d406da30 0x7f13d406fef0 unknown :-1 s=CLOSED pgs=4078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13e80febb0 0x7f13e81a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f13e81054a0 0x7f13e80fe510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 --2- 172.21.15.43:0/4159913924 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f13e81003d0 0x7f13e80fdfd0 unknown :-1 s=CLOSED pgs=3174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 >> 172.21.15.43:0/4159913924 conn(0x7f13e8077230 msgr2=0x7f13e8069850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 shutdown_connections 2024-02-05T08:08:37.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.470+0000 7f13ed165640 1 -- 172.21.15.43:0/4159913924 wait complete. 2024-02-05T08:08:37.482 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 159209 2024-02-05T08:08:37.482 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-02-05T08:08:37.482 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-02-05T08:08:37.482 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put three /etc/passwd 2024-02-05T08:08:37.521 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-02-05T08:08:37.522 INFO:tasks.workunit.client.0.smithi043.stderr:+ pp=f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 2024-02-05T08:08:37.522 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 12 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 -- 172.21.15.43:0/689419536 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 msgr2=0x7fc690100050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 --2- 172.21.15.43:0/689419536 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690100050 secure :-1 s=READY pgs=3177 cs=0 l=1 rev1=1 crypto rx=0x7fc684009890 tx=0x7fc68402bb70 comp rx=0 tx=0).stop 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 -- 172.21.15.43:0/689419536 shutdown_connections 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 --2- 172.21.15.43:0/689419536 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc690100590 0x7fc69010e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 --2- 172.21.15.43:0/689419536 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690100050 unknown :-1 s=CLOSED pgs=3177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 --2- 172.21.15.43:0/689419536 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc6901062d0 0x7fc6901066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 -- 172.21.15.43:0/689419536 >> 172.21.15.43:0/689419536 conn(0x7fc6900754b0 msgr2=0x7fc6900758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 -- 172.21.15.43:0/689419536 shutdown_connections 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.607+0000 7fc6968fd640 1 -- 172.21.15.43:0/689419536 wait complete. 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 Processor -- start 2024-02-05T08:08:37.610 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 -- start start 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc690100590 0x7fc690197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6901062d0 0x7fc69019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69019bf00 con 0x7fc6901062d0 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69019c070 con 0x7fc690100590 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc6968fd640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69019c230 con 0x7fc6900ffbd0 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc694e9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6901062d0 0x7fc69019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:44432/0 (socket says 172.21.15.43:44432) 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 -- 172.21.15.43:0/2980277539 learned_addr learned my addr 172.21.15.43:0/2980277539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68f7fe640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc690100590 0x7fc690197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 -- 172.21.15.43:0/2980277539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc690100590 msgr2=0x7fc690197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc690100590 0x7fc690197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 -- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6901062d0 msgr2=0x7fc69019b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:37.611 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6901062d0 0x7fc69019b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68ffff640 1 -- 172.21.15.43:0/2980277539 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc688009900 con 0x7fc6900ffbd0 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc694e9a640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6901062d0 0x7fc69019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.608+0000 7fc68f7fe640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc690100590 0x7fc690197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.609+0000 7fc68ffff640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690196e30 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 crypto rx=0x7fc68000d770 tx=0x7fc68000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.609+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680009c30 con 0x7fc6900ffbd0 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.609+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc68000de80 con 0x7fc6900ffbd0 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.609+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc684009490 con 0x7fc6900ffbd0 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.609+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc680004c70 con 0x7fc6900ffbd0 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.609+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6901a3bb0 con 0x7fc6900ffbd0 2024-02-05T08:08:37.612 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.610+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6900760e0 con 0x7fc6900ffbd0 2024-02-05T08:08:37.613 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.610+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fc680004dd0 con 0x7fc6900ffbd0 2024-02-05T08:08:37.613 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.611+0000 7fc68d7fa640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fc67806da30 0x7fc67806fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:37.613 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.611+0000 7fc68f7fe640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fc67806da30 0x7fc67806fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:37.613 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.611+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(783..783 src has 257..783) v4 ==== 9067+0+0 (secure 0 0 0) 0x7fc68008d9d0 con 0x7fc6900ffbd0 2024-02-05T08:08:37.613 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.611+0000 7fc68f7fe640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fc67806da30 0x7fc67806fef0 secure :-1 s=READY pgs=4080 cs=0 l=1 rev1=1 crypto rx=0x7fc690105ff0 tx=0x7fc684005890 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:37.615 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.615+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc6800567d0 con 0x7fc6900ffbd0 2024-02-05T08:08:37.799 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:37.798+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12} v 0) v1 -- 0x7fc69004f070 con 0x7fc6900ffbd0 2024-02-05T08:08:37.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:37 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4159913924' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:37.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:37 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:37.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:37 smithi043 ceph-mon[29879]: pgmap v1730: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:37.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:37 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4159913924' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:37.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:37 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:37.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:37 smithi043 ceph-mon[41444]: pgmap v1730: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:37 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4159913924' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:37 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:08:37.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:37 smithi050 ceph-mon[33492]: pgmap v1730: 176 pgs: 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:38.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.472+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]=0 pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' created v784) v1 ==== 176+0+0 (secure 0 0 0) 0x7fc6800200c0 con 0x7fc6900ffbd0 2024-02-05T08:08:38.585 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.585+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12} v 0) v1 -- 0x7fc690110960 con 0x7fc6900ffbd0 2024-02-05T08:08:38.586 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.586+0000 7fc68d7fa640 1 -- 172.21.15.43:0/2980277539 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]=0 pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' already exists v784) v1 ==== 183+0+0 (secure 0 0 0) 0x7fc680017030 con 0x7fc6900ffbd0 2024-02-05T08:08:38.586 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' already exists 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fc67806da30 msgr2=0x7fc67806fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fc67806da30 0x7fc67806fef0 secure :-1 s=READY pgs=4080 cs=0 l=1 rev1=1 crypto rx=0x7fc690105ff0 tx=0x7fc684005890 comp rx=0 tx=0).stop 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 msgr2=0x7fc690196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690196e30 secure :-1 s=READY pgs=3178 cs=0 l=1 rev1=1 crypto rx=0x7fc68000d770 tx=0x7fc68000dc30 comp rx=0 tx=0).stop 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 shutdown_connections 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fc67806da30 0x7fc67806fef0 unknown :-1 s=CLOSED pgs=4080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6901062d0 0x7fc69019b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.588 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fc690100590 0x7fc690197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 --2- 172.21.15.43:0/2980277539 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fc6900ffbd0 0x7fc690196e30 unknown :-1 s=CLOSED pgs=3178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 >> 172.21.15.43:0/2980277539 conn(0x7fc6900754b0 msgr2=0x7fc6900feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:38.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 shutdown_connections 2024-02-05T08:08:38.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.588+0000 7fc6968fd640 1 -- 172.21.15.43:0/2980277539 wait complete. 2024-02-05T08:08:38.597 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 rados 2024-02-05T08:08:38.681 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.680+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/3739616047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4102de0 msgr2=0x7f6ba410b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:38.681 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.680+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/3739616047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4102de0 0x7f6ba410b2c0 secure :-1 s=READY pgs=3015 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009860 tx=0x7f6b9802bc00 comp rx=0 tx=0).stop 2024-02-05T08:08:38.681 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.680+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/3739616047 shutdown_connections 2024-02-05T08:08:38.681 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.680+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/3739616047 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba410b890 0x7f6ba410dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.682 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/3739616047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4102de0 0x7f6ba410b2c0 unknown :-1 s=CLOSED pgs=3015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.682 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/3739616047 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba41024c0 0x7f6ba41028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.682 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/3739616047 >> 172.21.15.43:0/3739616047 conn(0x7f6ba40764b0 msgr2=0x7f6ba40768c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:38.682 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/3739616047 shutdown_connections 2024-02-05T08:08:38.682 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/3739616047 wait complete. 2024-02-05T08:08:38.682 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 Processor -- start 2024-02-05T08:08:38.682 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 -- start start 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba41024c0 0x7f6ba4100110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba4102de0 0x7f6ba4100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba410b890 0x7f6ba41a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba4100fc0 con 0x7f6ba410b890 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba41a5540 con 0x7f6ba41024c0 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.681+0000 7f6ba8a8a640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba41a5700 con 0x7f6ba4102de0 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba4102de0 0x7f6ba4100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba410b890 0x7f6ba41a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba4102de0 0x7f6ba4100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:44448/0 (socket says 172.21.15.43:44448) 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 -- 172.21.15.43:0/1123345055 learned_addr learned my addr 172.21.15.43:0/1123345055 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba259c640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba41024c0 0x7f6ba4100110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 -- 172.21.15.43:0/1123345055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba41024c0 msgr2=0x7f6ba4100110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:38.685 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba41024c0 0x7f6ba4100110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 -- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba410b890 msgr2=0x7f6ba41a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba410b890 0x7f6ba41a4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 -- 172.21.15.43:0/1123345055 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c009560 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba259c640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba41024c0 0x7f6ba4100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba2d9d640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba410b890 0x7f6ba41a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba4102de0 0x7f6ba4100650 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 crypto rx=0x7f6b98037040 tx=0x7f6b980346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9802c7a0 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b98009490 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6b980044d0 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9803d3f0 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.682+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba41a5cf0 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.683+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba4076fd0 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.684+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f6b9803b040 con 0x7f6ba4102de0 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.684+0000 7f6b937fe640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b8c06da30 0x7f6b8c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:38.686 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.684+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(784..784 src has 257..784) v4 ==== 9442+0+0 (secure 0 0 0) 0x7f6b980afc90 con 0x7f6ba4102de0 2024-02-05T08:08:38.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.684+0000 7f6ba259c640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b8c06da30 0x7f6b8c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:38.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.685+0000 7f6ba259c640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b8c06da30 0x7f6b8c06fef0 secure :-1 s=READY pgs=4081 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005fd0 tx=0x7f6b9c009260 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:38.689 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.689+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b980e3d20 con 0x7f6ba4102de0 2024-02-05T08:08:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:08:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[29879]: osdmap e783: 8 total, 8 up, 8 in 2024-02-05T08:08:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2980277539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:38.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:08:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[41444]: osdmap e783: 8 total, 8 up, 8 in 2024-02-05T08:08:38.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2980277539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:38.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:38 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:08:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:38 smithi050 ceph-mon[33492]: osdmap e783: 8 total, 8 up, 8 in 2024-02-05T08:08:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:38 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2980277539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:38.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:38 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:38.873 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:38.872+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"} v 0) v1 -- 0x7f6ba404f070 con 0x7f6ba4102de0 2024-02-05T08:08:39.475 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:08:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:39.481 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:39.481+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]=0 enabled application 'rados' on pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' v785) v1 ==== 213+0+0 (secure 0 0 0) 0x7f6b98078970 con 0x7f6ba4102de0 2024-02-05T08:08:39.599 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:39.599+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"} v 0) v1 -- 0x7f6ba4106090 con 0x7f6ba4102de0 2024-02-05T08:08:39.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]': finished 2024-02-05T08:08:39.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[29879]: osdmap e784: 8 total, 8 up, 8 in 2024-02-05T08:08:39.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/2980277539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:39.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:39.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1123345055' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:39.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:39.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[29879]: pgmap v1733: 188 pgs: 12 unknown, 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]': finished 2024-02-05T08:08:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[41444]: osdmap e784: 8 total, 8 up, 8 in 2024-02-05T08:08:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/2980277539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1123345055' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:39.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:39 smithi043 ceph-mon[41444]: pgmap v1733: 188 pgs: 12 unknown, 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]': finished 2024-02-05T08:08:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:39 smithi050 ceph-mon[33492]: osdmap e784: 8 total, 8 up, 8 in 2024-02-05T08:08:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/2980277539' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pg_num": 12}]: dispatch 2024-02-05T08:08:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:39 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1123345055' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:39 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:39.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:39 smithi050 ceph-mon[33492]: pgmap v1733: 188 pgs: 12 unknown, 176 active+clean; 717 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:40.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.485+0000 7f6b937fe640 1 -- 172.21.15.43:0/1123345055 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]=0 enabled application 'rados' on pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' v786) v1 ==== 213+0+0 (secure 0 0 0) 0x7f6b98039020 con 0x7f6ba4102de0 2024-02-05T08:08:40.486 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' 2024-02-05T08:08:40.488 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b8c06da30 msgr2=0x7f6b8c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b8c06da30 0x7f6b8c06fef0 secure :-1 s=READY pgs=4081 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c005fd0 tx=0x7f6b9c009260 comp rx=0 tx=0).stop 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba4102de0 msgr2=0x7f6ba4100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba4102de0 0x7f6ba4100650 secure :-1 s=READY pgs=3179 cs=0 l=1 rev1=1 crypto rx=0x7f6b98037040 tx=0x7f6b980346f0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 shutdown_connections 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b8c06da30 0x7f6b8c06fef0 unknown :-1 s=CLOSED pgs=4081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba410b890 0x7f6ba41a4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6ba4102de0 0x7f6ba4100650 unknown :-1 s=CLOSED pgs=3179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 --2- 172.21.15.43:0/1123345055 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6ba41024c0 0x7f6ba4100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 >> 172.21.15.43:0/1123345055 conn(0x7f6ba40764b0 msgr2=0x7f6ba4103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 shutdown_connections 2024-02-05T08:08:40.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.488+0000 7f6ba8a8a640 1 -- 172.21.15.43:0/1123345055 wait complete. 2024-02-05T08:08:40.500 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 max_objects 10 2024-02-05T08:08:40.594 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 -- 172.21.15.43:0/1799272156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 msgr2=0x7fa164106660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:40.594 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 --2- 172.21.15.43:0/1799272156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa164106660 secure :-1 s=READY pgs=3180 cs=0 l=1 rev1=1 crypto rx=0x7fa154009900 tx=0x7fa15402bc80 comp rx=0 tx=0).stop 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 -- 172.21.15.43:0/1799272156 shutdown_connections 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 --2- 172.21.15.43:0/1799272156 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100dd0 0x7fa16410cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 --2- 172.21.15.43:0/1799272156 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa164100410 0x7fa164100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 --2- 172.21.15.43:0/1799272156 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa164106660 unknown :-1 s=CLOSED pgs=3180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 -- 172.21.15.43:0/1799272156 >> 172.21.15.43:0/1799272156 conn(0x7fa164076f50 msgr2=0x7fa164077360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 -- 172.21.15.43:0/1799272156 shutdown_connections 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.591+0000 7fa169cfd640 1 -- 172.21.15.43:0/1799272156 wait complete. 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 Processor -- start 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 -- start start 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100410 0x7fa164197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa164100dd0 0x7fa1641983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa1641a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641a3260 con 0x7fa164100410 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641a33a0 con 0x7fa164100dd0 2024-02-05T08:08:40.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa169cfd640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1641a3560 con 0x7fa164106280 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa1637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100410 0x7fa164197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa163fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa1641a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa1637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100410 0x7fa164197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41450/0 (socket says 172.21.15.43:41450) 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa1637fe640 1 -- 172.21.15.43:0/3473125640 learned_addr learned my addr 172.21.15.43:0/3473125640 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa163fff640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa1641a2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:44476/0 (socket says 172.21.15.43:44476) 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.592+0000 7fa162ffd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa164100dd0 0x7fa1641983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa1637fe640 1 -- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 msgr2=0x7fa1641a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa1637fe640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa1641a2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa1637fe640 1 -- 172.21.15.43:0/3473125640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa164100dd0 msgr2=0x7fa1641983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa1637fe640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa164100dd0 0x7fa1641983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa1637fe640 1 -- 172.21.15.43:0/3473125640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c009840 con 0x7fa164100410 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa163fff640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa1641a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa1637fe640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100410 0x7fa164197e80 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7fa154006010 tx=0x7fa1540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:40.596 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540044e0 con 0x7fa164100410 2024-02-05T08:08:40.597 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa154009560 con 0x7fa164100410 2024-02-05T08:08:40.597 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa154003c80 con 0x7fa164100410 2024-02-05T08:08:40.597 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1540053d0 con 0x7fa164100410 2024-02-05T08:08:40.597 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.593+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1641a3bb0 con 0x7fa164100410 2024-02-05T08:08:40.597 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.594+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa164077a70 con 0x7fa164100410 2024-02-05T08:08:40.598 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.595+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7fa154005530 con 0x7fa164100410 2024-02-05T08:08:40.598 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.595+0000 7fa160ff9640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fa14c06da60 0x7fa14c06ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:08:40.598 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.595+0000 7fa162ffd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fa14c06da60 0x7fa14c06ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:08:40.598 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.595+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(786..786 src has 257..786) v4 ==== 9455+0+0 (secure 0 0 0) 0x7fa1540b0420 con 0x7fa164100410 2024-02-05T08:08:40.598 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.596+0000 7fa162ffd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fa14c06da60 0x7fa14c06ff20 secure :-1 s=READY pgs=4082 cs=0 l=1 rev1=1 crypto rx=0x7fa16406a610 tx=0x7fa15800a2d0 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:08:40.600 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.600+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa154079110 con 0x7fa164100410 2024-02-05T08:08:40.783 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:40.783+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa16404f070 con 0x7fa164100410 2024-02-05T08:08:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[29879]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:08:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]': finished 2024-02-05T08:08:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[29879]: osdmap e785: 8 total, 8 up, 8 in 2024-02-05T08:08:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/1123345055' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[41444]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:08:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]': finished 2024-02-05T08:08:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[41444]: osdmap e785: 8 total, 8 up, 8 in 2024-02-05T08:08:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/1123345055' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:40.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:40 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:40 smithi050 ceph-mon[33492]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:08:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]': finished 2024-02-05T08:08:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:40 smithi050 ceph-mon[33492]: osdmap e785: 8 total, 8 up, 8 in 2024-02-05T08:08:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:40 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/1123345055' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:40.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:40 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]: dispatch 2024-02-05T08:08:41.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:41.486+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 v787) v1 ==== 223+0+0 (secure 0 0 0) 0x7fa15407c7d0 con 0x7fa164100410 2024-02-05T08:08:41.603 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:41.603+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa1640690a0 con 0x7fa164100410 2024-02-05T08:08:41.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]': finished 2024-02-05T08:08:41.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[29879]: osdmap e786: 8 total, 8 up, 8 in 2024-02-05T08:08:41.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:08:41.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[29879]: pgmap v1736: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:08:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]': finished 2024-02-05T08:08:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[41444]: osdmap e786: 8 total, 8 up, 8 in 2024-02-05T08:08:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:08:41.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:41 smithi043 ceph-mon[41444]: pgmap v1736: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:08:41.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:41 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "app": "rados"}]': finished 2024-02-05T08:08:41.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:41 smithi050 ceph-mon[33492]: osdmap e786: 8 total, 8 up, 8 in 2024-02-05T08:08:41.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:41 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:08:41.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:41 smithi050 ceph-mon[33492]: pgmap v1736: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:08:42.488 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.488+0000 7fa160ff9640 1 -- 172.21.15.43:0/3473125640 <== mon.0 v2:172.21.15.43:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 v788) v1 ==== 223+0+0 (secure 0 0 0) 0x7fa1540430d0 con 0x7fa164100410 2024-02-05T08:08:42.488 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 2024-02-05T08:08:42.492 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fa14c06da60 msgr2=0x7fa14c06ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fa14c06da60 0x7fa14c06ff20 secure :-1 s=READY pgs=4082 cs=0 l=1 rev1=1 crypto rx=0x7fa16406a610 tx=0x7fa15800a2d0 comp rx=0 tx=0).stop 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100410 msgr2=0x7fa164197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100410 0x7fa164197e80 secure :-1 s=READY pgs=3016 cs=0 l=1 rev1=1 crypto rx=0x7fa154006010 tx=0x7fa1540042f0 comp rx=0 tx=0).stop 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 shutdown_connections 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7fa14c06da60 0x7fa14c06ff20 unknown :-1 s=CLOSED pgs=4082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7fa164106280 0x7fa1641a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7fa164100dd0 0x7fa1641983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 --2- 172.21.15.43:0/3473125640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa164100410 0x7fa164197e80 unknown :-1 s=CLOSED pgs=3016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 >> 172.21.15.43:0/3473125640 conn(0x7fa164076f50 msgr2=0x7fa164103d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 shutdown_connections 2024-02-05T08:08:42.493 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:08:42.492+0000 7fa169cfd640 1 -- 172.21.15.43:0/3473125640 wait complete. 2024-02-05T08:08:42.507 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-02-05T08:08:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:08:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[29879]: osdmap e787: 8 total, 8 up, 8 in 2024-02-05T08:08:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:08:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:08:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[41444]: osdmap e787: 8 total, 8 up, 8 in 2024-02-05T08:08:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:08:42.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:42 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:08:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:42 smithi050 ceph-mon[33492]: osdmap e787: 8 total, 8 up, 8 in 2024-02-05T08:08:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:42 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]: dispatch 2024-02-05T08:08:42.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:42 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:42.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:08:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:08:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:08:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:43 smithi043 ceph-mon[29879]: osdmap e788: 8 total, 8 up, 8 in 2024-02-05T08:08:43.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:43 smithi043 ceph-mon[29879]: pgmap v1739: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:08:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:08:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:43 smithi043 ceph-mon[41444]: osdmap e788: 8 total, 8 up, 8 in 2024-02-05T08:08:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:43 smithi043 ceph-mon[41444]: pgmap v1739: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:08:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3473125640' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "field": "max_objects", "val": "10"}]': finished 2024-02-05T08:08:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:43 smithi050 ceph-mon[33492]: osdmap e788: 8 total, 8 up, 8 in 2024-02-05T08:08:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:43 smithi050 ceph-mon[33492]: pgmap v1739: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-05T08:08:45.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:45 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:45.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:45 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:45 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:45.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:45 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:45 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:08:45.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:45 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:46.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:46 smithi043 ceph-mon[29879]: pgmap v1740: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 874 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T08:08:46.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:46 smithi043 ceph-mon[41444]: pgmap v1740: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 874 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T08:08:46.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:46 smithi050 ceph-mon[33492]: pgmap v1740: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 874 B/s rd, 4.1 KiB/s wr, 1 op/s 2024-02-05T08:08:47.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:47 smithi043 ceph-mon[29879]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:08:47.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:47 smithi043 ceph-mon[41444]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:08:47.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:47 smithi050 ceph-mon[33492]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-05T08:08:48.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:48 smithi043 ceph-mon[29879]: pgmap v1741: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-05T08:08:48.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:48 smithi043 ceph-mon[41444]: pgmap v1741: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-05T08:08:48.572 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:48 smithi050 ceph-mon[33492]: pgmap v1741: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 1 op/s 2024-02-05T08:08:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:08:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:08:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:50.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:50 smithi043 ceph-mon[41444]: pgmap v1742: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:08:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:50 smithi043 ceph-mon[29879]: pgmap v1742: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:08:50.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:50 smithi050 ceph-mon[33492]: pgmap v1742: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:08:52.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:52 smithi043 ceph-mon[29879]: pgmap v1743: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:08:52.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:52 smithi043 ceph-mon[41444]: pgmap v1743: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:08:52.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:52 smithi050 ceph-mon[33492]: pgmap v1743: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-05T08:08:52.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:08:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:08:53.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:53.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:53.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:08:54.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:54 smithi043 ceph-mon[29879]: pgmap v1744: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 944 B/s rd, 0 op/s 2024-02-05T08:08:54.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:54 smithi043 ceph-mon[41444]: pgmap v1744: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 944 B/s rd, 0 op/s 2024-02-05T08:08:54.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:54 smithi050 ceph-mon[33492]: pgmap v1744: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 944 B/s rd, 0 op/s 2024-02-05T08:08:56.814 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:56 smithi043 ceph-mon[29879]: pgmap v1745: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:56.814 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:56 smithi043 ceph-mon[41444]: pgmap v1745: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:56.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:56 smithi050 ceph-mon[33492]: pgmap v1745: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:08:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:58 smithi043 ceph-mon[29879]: pgmap v1746: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:58.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:58 smithi043 ceph-mon[41444]: pgmap v1746: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:58.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:58 smithi050 ceph-mon[33492]: pgmap v1746: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:08:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:08:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:59.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:08:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:08:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:08:59.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:08:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:08:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:08:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:00.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:00 smithi043 ceph-mon[29879]: pgmap v1747: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:00.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:00 smithi043 ceph-mon[41444]: pgmap v1747: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:00.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:00 smithi050 ceph-mon[33492]: pgmap v1747: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:02 smithi043 ceph-mon[29879]: pgmap v1748: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:02.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:02 smithi043 ceph-mon[41444]: pgmap v1748: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:02 smithi050 ceph-mon[33492]: pgmap v1748: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:02.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:09:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:09:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:03.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:04 smithi043 ceph-mon[29879]: pgmap v1749: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:04 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:09:04.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:04 smithi043 ceph-mon[41444]: pgmap v1749: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:04.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:04 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:09:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:04 smithi050 ceph-mon[33492]: pgmap v1749: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:04 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:09:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:05 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:09:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:05 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:09:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:05 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:09:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:05 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:09:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:05 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:09:05.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:05 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:09:05.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:05 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:09:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:05 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:09:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:05 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:09:06.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:06 smithi043 ceph-mon[29879]: pgmap v1750: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:06.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:06 smithi043 ceph-mon[41444]: pgmap v1750: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:06.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:06 smithi050 ceph-mon[33492]: pgmap v1750: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:07.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:07 smithi043 ceph-mon[29879]: pgmap v1751: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:07.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:07 smithi043 ceph-mon[41444]: pgmap v1751: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:07.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:07 smithi050 ceph-mon[33492]: pgmap v1751: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:09:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:09:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:10 smithi043 ceph-mon[29879]: pgmap v1752: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:10.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:10 smithi043 ceph-mon[41444]: pgmap v1752: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:10.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:10 smithi050 ceph-mon[33492]: pgmap v1752: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:12.509 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-02-05T08:09:12.510 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.510 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj1 /etc/passwd 2024-02-05T08:09:12.551 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.552 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj2 /etc/passwd 2024-02-05T08:09:12.590 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.590 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj3 /etc/passwd 2024-02-05T08:09:12.626 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.626 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj4 /etc/passwd 2024-02-05T08:09:12.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.661 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj5 /etc/passwd 2024-02-05T08:09:12.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj6 /etc/passwd 2024-02-05T08:09:12.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj7 /etc/passwd 2024-02-05T08:09:12.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj8 /etc/passwd 2024-02-05T08:09:12.799 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.799 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj9 /etc/passwd 2024-02-05T08:09:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:12 smithi043 ceph-mon[29879]: pgmap v1753: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:12.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:12 smithi043 ceph-mon[41444]: pgmap v1753: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:12 smithi050 ceph-mon[33492]: pgmap v1753: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:12.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:09:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:09:12.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-02-05T08:09:12.834 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 put obj10 /etc/passwd 2024-02-05T08:09:12.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-02-05T08:09:13.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:13.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:14 smithi043 ceph-mon[29879]: pgmap v1754: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:14.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:14 smithi043 ceph-mon[41444]: pgmap v1754: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:14.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:14 smithi050 ceph-mon[33492]: pgmap v1754: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:16 smithi043 ceph-mon[29879]: pgmap v1755: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:16.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:16 smithi043 ceph-mon[41444]: pgmap v1755: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:16.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:16 smithi050 ceph-mon[33492]: pgmap v1755: 188 pgs: 188 active+clean; 740 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:18.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:18 smithi043 ceph-mon[29879]: pgmap v1756: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:09:18.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:18 smithi043 ceph-mon[41444]: pgmap v1756: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:09:18.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:18 smithi050 ceph-mon[33492]: pgmap v1756: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:09:19.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:09:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:09:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:20.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:20 smithi043 ceph-mon[29879]: pgmap v1757: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-05T08:09:20.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:20 smithi043 ceph-mon[41444]: pgmap v1757: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-05T08:09:20.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:20 smithi050 ceph-mon[33492]: pgmap v1757: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-02-05T08:09:21.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:21 smithi043 ceph-mon[29879]: pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' is full (reached quota's max_objects: 10) 2024-02-05T08:09:21.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:21 smithi043 ceph-mon[29879]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:09:21.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:21 smithi043 ceph-mon[29879]: osdmap e789: 8 total, 8 up, 8 in 2024-02-05T08:09:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:21 smithi043 ceph-mon[41444]: pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' is full (reached quota's max_objects: 10) 2024-02-05T08:09:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:21 smithi043 ceph-mon[41444]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:09:21.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:21 smithi043 ceph-mon[41444]: osdmap e789: 8 total, 8 up, 8 in 2024-02-05T08:09:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:21 smithi050 ceph-mon[33492]: pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' is full (reached quota's max_objects: 10) 2024-02-05T08:09:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:21 smithi050 ceph-mon[33492]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-05T08:09:21.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:21 smithi050 ceph-mon[33492]: osdmap e789: 8 total, 8 up, 8 in 2024-02-05T08:09:22.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:22 smithi043 ceph-mon[41444]: pgmap v1758: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:09:22.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:22 smithi043 ceph-mon[29879]: pgmap v1758: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:09:22.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:22 smithi050 ceph-mon[33492]: pgmap v1758: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-05T08:09:22.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:09:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:09:23.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:23 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:23.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:23 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:23.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:23 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:24.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:24 smithi043 ceph-mon[29879]: pgmap v1760: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:09:24.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:24 smithi043 ceph-mon[41444]: pgmap v1760: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:09:24.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:24 smithi050 ceph-mon[33492]: pgmap v1760: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:09:26.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:26 smithi043 ceph-mon[29879]: pgmap v1761: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:09:26.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:26 smithi043 ceph-mon[41444]: pgmap v1761: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:09:26.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:26 smithi050 ceph-mon[33492]: pgmap v1761: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-05T08:09:27.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:27 smithi043 ceph-mon[29879]: pgmap v1762: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:27.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:27 smithi043 ceph-mon[41444]: pgmap v1762: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:27.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:27 smithi050 ceph-mon[33492]: pgmap v1762: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:29.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:09:29 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:09:29] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:30.525 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:30 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:09:30.525 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:30 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:30.525 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:30 smithi050 ceph-mon[33492]: pgmap v1763: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:30.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:30 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:09:30.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:30 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:30.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:30 smithi043 ceph-mon[29879]: pgmap v1763: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:30 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:09:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:30 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:30.563 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:30 smithi043 ceph-mon[41444]: pgmap v1763: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:32.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:32 smithi043 ceph-mon[29879]: pgmap v1764: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:32.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:32 smithi043 ceph-mon[41444]: pgmap v1764: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:32.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:32 smithi050 ceph-mon[33492]: pgmap v1764: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-05T08:09:32.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:09:32 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:09:33.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:33 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:33.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:33 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:33.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:33 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:34.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:34 smithi043 ceph-mon[29879]: pgmap v1765: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-05T08:09:34.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:34 smithi043 ceph-mon[41444]: pgmap v1765: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-05T08:09:34.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:34 smithi050 ceph-mon[33492]: pgmap v1765: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 858 B/s rd, 0 op/s 2024-02-05T08:09:36.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:36 smithi043 ceph-mon[41444]: pgmap v1766: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:36.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:36 smithi043 ceph-mon[29879]: pgmap v1766: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:36.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:36 smithi050 ceph-mon[33492]: pgmap v1766: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:38.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:38 smithi043 ceph-mon[41444]: pgmap v1767: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:38.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:38 smithi043 ceph-mon[29879]: pgmap v1767: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:38.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:38 smithi050 ceph-mon[33492]: pgmap v1767: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:39.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:09:39 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:09:39] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:40.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:40 smithi043 ceph-mon[41444]: pgmap v1768: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:40.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:40 smithi043 ceph-mon[29879]: pgmap v1768: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:40.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:40 smithi050 ceph-mon[33492]: pgmap v1768: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:42.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:42 smithi043 ceph-mon[29879]: pgmap v1769: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:42.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:42 smithi043 ceph-mon[41444]: pgmap v1769: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:42.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:42 smithi050 ceph-mon[33492]: pgmap v1769: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:09:42.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:09:42 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:09:42.871 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p e2a6ede4-caf1-4034-97c2-ddac1835f21e put threemore /etc/passwd 2024-02-05T08:09:42.913 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota e2a6ede4-caf1-4034-97c2-ddac1835f21e max_bytes 0 2024-02-05T08:09:43.005 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 -- 172.21.15.43:0/356776693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac100520 msgr2=0x7f68ac10e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:43.005 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 --2- 172.21.15.43:0/356776693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac100520 0x7f68ac10e120 secure :-1 s=READY pgs=3030 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009cd0 tx=0x7f68a402c180 comp rx=0 tx=0).stop 2024-02-05T08:09:43.005 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 -- 172.21.15.43:0/356776693 shutdown_connections 2024-02-05T08:09:43.005 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 --2- 172.21.15.43:0/356776693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac100520 0x7f68ac10e120 unknown :-1 s=CLOSED pgs=3030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:43.005 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 --2- 172.21.15.43:0/356776693 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f68ac0ffb60 0x7f68ac0fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 --2- 172.21.15.43:0/356776693 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68ac1074a0 0x7f68ac107880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 -- 172.21.15.43:0/356776693 >> 172.21.15.43:0/356776693 conn(0x7f68ac0754e0 msgr2=0x7f68ac0758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 -- 172.21.15.43:0/356776693 shutdown_connections 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.002+0000 7f68b31af640 1 -- 172.21.15.43:0/356776693 wait complete. 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 Processor -- start 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 -- start start 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f68ac0ffb60 0x7f68ac196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68ac100520 0x7f68ac197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac1074a0 0x7f68ac19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:43.006 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19bf10 con 0x7f68ac1074a0 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19c080 con 0x7f68ac100520 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b31af640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac19c240 con 0x7f68ac0ffb60 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac1074a0 0x7f68ac19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b0f4b640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f68ac0ffb60 0x7f68ac196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac1074a0 0x7f68ac19b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38358/0 (socket says 172.21.15.43:38358) 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 -- 172.21.15.43:0/3719148819 learned_addr learned my addr 172.21.15.43:0/3719148819 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68abfff640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68ac100520 0x7f68ac197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 -- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f68ac0ffb60 msgr2=0x7f68ac196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f68ac0ffb60 0x7f68ac196e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:43.007 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 -- 172.21.15.43:0/3719148819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68ac100520 msgr2=0x7f68ac197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68ac100520 0x7f68ac197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898009490 con 0x7f68ac1074a0 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b0f4b640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f68ac0ffb60 0x7f68ac196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68abfff640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68ac100520 0x7f68ac197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.003+0000 7f68b174c640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac1074a0 0x7f68ac19b910 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009ca0 tx=0x7f68a40346e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.004+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4003d80 con 0x7f68ac1074a0 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.004+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a4009900 con 0x7f68ac1074a0 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.004+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f68a4032bc0 con 0x7f68ac1074a0 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.004+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40044c0 con 0x7f68ac1074a0 2024-02-05T08:09:43.008 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.004+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac1a3bc0 con 0x7f68ac1074a0 2024-02-05T08:09:43.009 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.005+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac0760c0 con 0x7f68ac1074a0 2024-02-05T08:09:43.009 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.005+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f68a403b040 con 0x7f68ac1074a0 2024-02-05T08:09:43.009 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.006+0000 7f68a9ffb640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f689c06da30 0x7f689c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:43.010 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.006+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(789..789 src has 257..789) v4 ==== 9455+0+0 (secure 0 0 0) 0x7f68a40b0b80 con 0x7f68ac1074a0 2024-02-05T08:09:43.010 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.006+0000 7f68b0f4b640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f689c06da30 0x7f689c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:43.010 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.006+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7f689c079ce0 con 0x7f68ac1074a0 2024-02-05T08:09:43.010 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.006+0000 7f68b0f4b640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f689c06da30 0x7f689c06fef0 secure :-1 s=READY pgs=4094 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009d60 tx=0x7f68a0009110 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:09:43.010 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.010+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f68a40797b0 con 0x7f68ac1074a0 2024-02-05T08:09:43.195 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.195+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f68ac04f070 con 0x7f68ac1074a0 2024-02-05T08:09:43.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.447+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v790) v1 ==== 217+0+0 (secure 0 0 0) 0x7f68a40430d0 con 0x7f68ac1074a0 2024-02-05T08:09:43.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.448+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(790..790 src has 257..790) v4 ==== 628+0+0 (secure 0 0 0) 0x7f68a4032d30 con 0x7f68ac1074a0 2024-02-05T08:09:43.448 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.448+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=791}) v3 -- 0x7f689c07acc0 con 0x7f68ac1074a0 2024-02-05T08:09:43.565 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:43.565+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f68ac04f070 con 0x7f68ac1074a0 2024-02-05T08:09:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:43 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:43.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:43 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:09:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:43 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:43.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:43 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:09:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:43 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:43.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:43 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:09:44.450 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.449+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v791) v1 ==== 217+0+0 (secure 0 0 0) 0x7f68a4039020 con 0x7f68ac1074a0 2024-02-05T08:09:44.450 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:09:44.450 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.450+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 <== mon.0 v2:172.21.15.43:3300/0 10 ==== osd_map(791..791 src has 257..791) v4 ==== 628+0+0 (secure 0 0 0) 0x7f68a407dc10 con 0x7f68ac1074a0 2024-02-05T08:09:44.450 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.450+0000 7f68a9ffb640 1 -- 172.21.15.43:0/3719148819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7f689c07ad40 con 0x7f68ac1074a0 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f689c06da30 msgr2=0x7f689c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f689c06da30 0x7f689c06fef0 secure :-1 s=READY pgs=4094 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009d60 tx=0x7f68a0009110 comp rx=0 tx=0).stop 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac1074a0 msgr2=0x7f68ac19b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac1074a0 0x7f68ac19b910 secure :-1 s=READY pgs=3031 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009ca0 tx=0x7f68a40346e0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 shutdown_connections 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f689c06da30 0x7f689c06fef0 unknown :-1 s=CLOSED pgs=4094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68ac1074a0 0x7f68ac19b910 unknown :-1 s=CLOSED pgs=3031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f68ac100520 0x7f68ac197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 --2- 172.21.15.43:0/3719148819 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f68ac0ffb60 0x7f68ac196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 >> 172.21.15.43:0/3719148819 conn(0x7f68ac0754e0 msgr2=0x7f68ac104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:09:44.455 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.454+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 shutdown_connections 2024-02-05T08:09:44.456 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.455+0000 7f68b31af640 1 -- 172.21.15.43:0/3719148819 wait complete. 2024-02-05T08:09:44.469 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota e2a6ede4-caf1-4034-97c2-ddac1835f21e max_objects 0 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.557+0000 7f0ffeff7640 1 -- 172.21.15.43:0/46975139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff810b890 msgr2=0x7f0ff810dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.557+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/46975139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff810b890 0x7f0ff810dcc0 secure :-1 s=READY pgs=3032 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009860 tx=0x7f0fe802bc00 comp rx=0 tx=0).stop 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- 172.21.15.43:0/46975139 shutdown_connections 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/46975139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff810b890 0x7f0ff810dcc0 unknown :-1 s=CLOSED pgs=3032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/46975139 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ff8102de0 0x7f0ff810b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/46975139 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ff81024c0 0x7f0ff81028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- 172.21.15.43:0/46975139 >> 172.21.15.43:0/46975139 conn(0x7f0ff80764b0 msgr2=0x7f0ff80768c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- 172.21.15.43:0/46975139 shutdown_connections 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- 172.21.15.43:0/46975139 wait complete. 2024-02-05T08:09:44.560 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 Processor -- start 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- start start 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ff81024c0 0x7f0ff8196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff8102de0 0x7f0ff81972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ff810b890 0x7f0ff819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff819be70 con 0x7f0ff8102de0 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff819bfe0 con 0x7f0ff81024c0 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffeff7640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff819c1a0 con 0x7f0ff810b890 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ff7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff8102de0 0x7f0ff81972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ffd594640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ff810b890 0x7f0ff819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ff7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff8102de0 0x7f0ff81972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38374/0 (socket says 172.21.15.43:38374) 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.558+0000 7f0ff7fff640 1 -- 172.21.15.43:0/609125479 learned_addr learned my addr 172.21.15.43:0/609125479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ffcd93640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ff81024c0 0x7f0ff8196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:44.561 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff7fff640 1 -- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ff810b890 msgr2=0x7f0ff819b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff7fff640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ff810b890 0x7f0ff819b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff7fff640 1 -- 172.21.15.43:0/609125479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ff81024c0 msgr2=0x7f0ff8196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff7fff640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ff81024c0 0x7f0ff8196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff7fff640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009900 con 0x7f0ff8102de0 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ffd594640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ff810b890 0x7f0ff819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff7fff640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff8102de0 0x7f0ff81972e0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00d950 tx=0x7f0fec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec009c30 con 0x7f0ff8102de0 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0fec009d90 con 0x7f0ff8102de0 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe8009490 con 0x7f0ff8102de0 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00f620 con 0x7f0ff8102de0 2024-02-05T08:09:44.562 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.559+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff81a3b20 con 0x7f0ff8102de0 2024-02-05T08:09:44.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.560+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff8076fd0 con 0x7f0ff8102de0 2024-02-05T08:09:44.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.561+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f0fec010460 con 0x7f0ff8102de0 2024-02-05T08:09:44.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.561+0000 7f0ff5ffb640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f0fe006da30 0x7f0fe006fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:09:44.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.561+0000 7f0ffcd93640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f0fe006da30 0x7f0fe006fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:09:44.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.561+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(791..791 src has 257..791) v4 ==== 9455+0+0 (secure 0 0 0) 0x7f0fec08dc30 con 0x7f0ff8102de0 2024-02-05T08:09:44.563 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.561+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7f0fe0079d50 con 0x7f0ff8102de0 2024-02-05T08:09:44.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.562+0000 7f0ffcd93640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f0fe006da30 0x7f0fe006fef0 secure :-1 s=READY pgs=4095 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005fd0 tx=0x7f0ff0010040 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:09:44.566 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.566+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0fec020060 con 0x7f0ff8102de0 2024-02-05T08:09:44.750 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:44.750+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f0ff804f070 con 0x7f0ff8102de0 2024-02-05T08:09:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[29879]: pgmap v1770: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:09:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[29879]: osdmap e790: 8 total, 8 up, 8 in 2024-02-05T08:09:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:09:44.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:44.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[41444]: pgmap v1770: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:09:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[41444]: osdmap e790: 8 total, 8 up, 8 in 2024-02-05T08:09:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:09:44.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:44 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:44 smithi050 ceph-mon[33492]: pgmap v1770: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:09:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:09:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:44 smithi050 ceph-mon[33492]: osdmap e790: 8 total, 8 up, 8 in 2024-02-05T08:09:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:44 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-05T08:09:44.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:44 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:45.466 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:45.465+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v792) v1 ==== 221+0+0 (secure 0 0 0) 0x7f0fec020030 con 0x7f0ff8102de0 2024-02-05T08:09:45.466 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:45.466+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 8 ==== osd_map(792..792 src has 257..792) v4 ==== 628+0+0 (secure 0 0 0) 0x7f0fec05e790 con 0x7f0ff8102de0 2024-02-05T08:09:45.466 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:45.466+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7f0fe007acb0 con 0x7f0ff8102de0 2024-02-05T08:09:45.584 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:45.584+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f0ff8106090 con 0x7f0ff8102de0 2024-02-05T08:09:45.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:09:45.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[29879]: osdmap e791: 8 total, 8 up, 8 in 2024-02-05T08:09:45.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:09:45.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[29879]: pgmap v1773: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:09:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:09:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[41444]: osdmap e791: 8 total, 8 up, 8 in 2024-02-05T08:09:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:09:45.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:45 smithi043 ceph-mon[41444]: pgmap v1773: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:09:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3719148819' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_bytes", "val": "0"}]': finished 2024-02-05T08:09:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:45 smithi050 ceph-mon[33492]: osdmap e791: 8 total, 8 up, 8 in 2024-02-05T08:09:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:45 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:09:45.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:45 smithi050 ceph-mon[33492]: pgmap v1773: 188 pgs: 188 active+clean; 852 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:09:46.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.470+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e v793) v1 ==== 221+0+0 (secure 0 0 0) 0x7f0fec020020 con 0x7f0ff8102de0 2024-02-05T08:09:46.471 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool e2a6ede4-caf1-4034-97c2-ddac1835f21e 2024-02-05T08:09:46.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.470+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 <== mon.0 v2:172.21.15.43:3300/0 10 ==== osd_map(793..793 src has 257..793) v4 ==== 628+0+0 (secure 0 0 0) 0x7f0fec062d50 con 0x7f0ff8102de0 2024-02-05T08:09:46.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.471+0000 7f0ff5ffb640 1 -- 172.21.15.43:0/609125479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=794}) v3 -- 0x7f0fe007aec0 con 0x7f0ff8102de0 2024-02-05T08:09:46.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f0fe006da30 msgr2=0x7f0fe006fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:46.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f0fe006da30 0x7f0fe006fef0 secure :-1 s=READY pgs=4095 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0005fd0 tx=0x7f0ff0010040 comp rx=0 tx=0).stop 2024-02-05T08:09:46.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff8102de0 msgr2=0x7f0ff81972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:09:46.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff8102de0 0x7f0ff81972e0 secure :-1 s=READY pgs=3033 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00d950 tx=0x7f0fec00de10 comp rx=0 tx=0).stop 2024-02-05T08:09:46.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 shutdown_connections 2024-02-05T08:09:46.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f0fe006da30 0x7f0fe006fef0 unknown :-1 s=CLOSED pgs=4095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:46.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f0ff810b890 0x7f0ff819b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:46.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ff8102de0 0x7f0ff81972e0 unknown :-1 s=CLOSED pgs=3033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:46.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 --2- 172.21.15.43:0/609125479 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f0ff81024c0 0x7f0ff8196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:09:46.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 >> 172.21.15.43:0/609125479 conn(0x7f0ff80764b0 msgr2=0x7f0ff8103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:09:46.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.475+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 shutdown_connections 2024-02-05T08:09:46.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:09:46.478+0000 7f0ffeff7640 1 -- 172.21.15.43:0/609125479 wait complete. 2024-02-05T08:09:46.491 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-02-05T08:09:46.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:09:46.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:46 smithi043 ceph-mon[29879]: osdmap e792: 8 total, 8 up, 8 in 2024-02-05T08:09:46.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:46 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:09:46.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:09:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:46 smithi043 ceph-mon[41444]: osdmap e792: 8 total, 8 up, 8 in 2024-02-05T08:09:46.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:46 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:09:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:09:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:46 smithi050 ceph-mon[33492]: osdmap e792: 8 total, 8 up, 8 in 2024-02-05T08:09:46.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:46 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]: dispatch 2024-02-05T08:09:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:47 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:09:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:47 smithi043 ceph-mon[29879]: osdmap e793: 8 total, 8 up, 8 in 2024-02-05T08:09:47.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:47 smithi043 ceph-mon[29879]: pgmap v1776: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:09:47.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:47 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:09:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:47 smithi043 ceph-mon[41444]: osdmap e793: 8 total, 8 up, 8 in 2024-02-05T08:09:47.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:47 smithi043 ceph-mon[41444]: pgmap v1776: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:09:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:47 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/609125479' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "field": "max_objects", "val": "0"}]': finished 2024-02-05T08:09:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:47 smithi050 ceph-mon[33492]: osdmap e793: 8 total, 8 up, 8 in 2024-02-05T08:09:47.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:47 smithi050 ceph-mon[33492]: pgmap v1776: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-02-05T08:09:49.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:09:49 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:09:49] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:50.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:50 smithi043 ceph-mon[29879]: pgmap v1777: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 867 B/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T08:09:50.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:50 smithi043 ceph-mon[41444]: pgmap v1777: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 867 B/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T08:09:50.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:50 smithi050 ceph-mon[33492]: pgmap v1777: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 867 B/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-05T08:09:52.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:52 smithi043 ceph-mon[29879]: pgmap v1778: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-02-05T08:09:52.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:52 smithi043 ceph-mon[41444]: pgmap v1778: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-02-05T08:09:52.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:52 smithi050 ceph-mon[33492]: pgmap v1778: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 1 op/s 2024-02-05T08:09:52.824 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:09:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:09:53.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:53 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:53.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:53 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:53.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:53 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:09:54.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:54 smithi043 ceph-mon[41444]: pgmap v1779: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-05T08:09:54.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:54 smithi043 ceph-mon[29879]: pgmap v1779: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-05T08:09:54.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:54 smithi050 ceph-mon[33492]: pgmap v1779: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-05T08:09:56.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:56 smithi043 ceph-mon[29879]: pgmap v1780: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-05T08:09:56.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:56 smithi043 ceph-mon[41444]: pgmap v1780: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-05T08:09:56.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:56 smithi050 ceph-mon[33492]: pgmap v1780: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-02-05T08:09:58.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:58 smithi043 ceph-mon[41444]: pgmap v1781: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-02-05T08:09:58.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:58 smithi043 ceph-mon[29879]: pgmap v1781: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-02-05T08:09:58.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:58 smithi050 ceph-mon[33492]: pgmap v1781: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 940 B/s rd, 0 op/s 2024-02-05T08:09:59.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:09:59 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:59.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:09:59 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:09:59.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:09:59 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:09:59] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:09:59.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:09:59 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:00.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:00 smithi050 ceph-mon[33492]: pgmap v1782: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:00.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:00 smithi050 ceph-mon[33492]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-02-05T08:10:00.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:00 smithi050 ceph-mon[33492]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-05T08:10:00.723 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:00 smithi050 ceph-mon[33492]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:10:00.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:00 smithi050 ceph-mon[33492]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:10:00.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:00 smithi050 ceph-mon[33492]: [WRN] POOL_FULL: 1 pool(s) full 2024-02-05T08:10:00.724 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:00 smithi050 ceph-mon[33492]: pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' is full (running out of quota) 2024-02-05T08:10:00.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[41444]: pgmap v1782: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:00.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[41444]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-02-05T08:10:00.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[41444]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[41444]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[41444]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[41444]: [WRN] POOL_FULL: 1 pool(s) full 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[41444]: pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' is full (running out of quota) 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[29879]: pgmap v1782: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[29879]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[29879]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[29879]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[29879]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[29879]: [WRN] POOL_FULL: 1 pool(s) full 2024-02-05T08:10:00.813 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:00 smithi043 ceph-mon[29879]: pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' is full (running out of quota) 2024-02-05T08:10:02.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:02 smithi043 ceph-mon[29879]: pgmap v1783: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:02.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:02 smithi043 ceph-mon[41444]: pgmap v1783: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:02.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:02 smithi050 ceph-mon[33492]: pgmap v1783: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:02.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:10:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:10:03.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:03 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:03.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:03 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:03.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:03 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:04.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:04 smithi043 ceph-mon[29879]: pgmap v1784: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:04.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:04 smithi043 ceph-mon[41444]: pgmap v1784: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:04.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:04 smithi050 ceph-mon[33492]: pgmap v1784: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:05 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:10:05.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:05 smithi043 ceph-mon[29879]: pgmap v1785: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:05.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:05 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:10:05.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:05 smithi043 ceph-mon[41444]: pgmap v1785: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:05 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-05T08:10:05.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:05 smithi050 ceph-mon[33492]: pgmap v1785: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:07.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[29879]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:10:07.063 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:06 smithi043 ceph-mon[41444]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.071 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:06 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:06 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:06 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-05T08:10:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:06 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-05T08:10:07.072 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:06 smithi050 ceph-mon[33492]: from='mgr.24485 ' entity='mgr.y' 2024-02-05T08:10:08.062 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:07 smithi043 ceph-mon[29879]: pgmap v1786: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:08.062 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:07 smithi043 ceph-mon[41444]: pgmap v1786: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:08.071 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:07 smithi050 ceph-mon[33492]: pgmap v1786: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:09.562 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:09 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:10:09] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:10.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:10 smithi043 ceph-mon[29879]: pgmap v1787: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:10.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:10 smithi043 ceph-mon[41444]: pgmap v1787: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:10.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:10 smithi050 ceph-mon[33492]: pgmap v1787: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:12.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:12 smithi043 ceph-mon[29879]: pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:12.813 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:12 smithi043 ceph-mon[41444]: pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:12.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:12 smithi050 ceph-mon[33492]: pgmap v1788: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:12.822 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:10:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:10:13.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:13 smithi043 ceph-mon[29879]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:13.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:13 smithi043 ceph-mon[41444]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:13.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:13 smithi050 ceph-mon[33492]: from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-05T08:10:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:14 smithi043 ceph-mon[29879]: pgmap v1789: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:14.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:14 smithi043 ceph-mon[29879]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:14.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:14 smithi043 ceph-mon[41444]: pgmap v1789: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:14.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:14 smithi043 ceph-mon[41444]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:14.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:14 smithi050 ceph-mon[33492]: pgmap v1789: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:14.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:14 smithi050 ceph-mon[33492]: from='mgr.24485 172.21.15.43:0/1883085232' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-05T08:10:16.492 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete e2a6ede4-caf1-4034-97c2-ddac1835f21e e2a6ede4-caf1-4034-97c2-ddac1835f21e --yes-i-really-really-mean-it 2024-02-05T08:10:16.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 -- 172.21.15.43:0/3190605920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c1003a0 msgr2=0x7f6b4c100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:16.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 --2- 172.21.15.43:0/3190605920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c1003a0 0x7f6b4c100780 secure :-1 s=READY pgs=3034 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009890 tx=0x7f6b3c02bb70 comp rx=0 tx=0).stop 2024-02-05T08:10:16.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 -- 172.21.15.43:0/3190605920 shutdown_connections 2024-02-05T08:10:16.589 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 --2- 172.21.15.43:0/3190605920 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c105470 0x7f6b4c10d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 --2- 172.21.15.43:0/3190605920 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b4c100d50 0x7f6b4c104d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 --2- 172.21.15.43:0/3190605920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c1003a0 0x7f6b4c100780 unknown :-1 s=CLOSED pgs=3034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 -- 172.21.15.43:0/3190605920 >> 172.21.15.43:0/3190605920 conn(0x7f6b4c0764b0 msgr2=0x7f6b4c0768c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 -- 172.21.15.43:0/3190605920 shutdown_connections 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.586+0000 7f6b52385640 1 -- 172.21.15.43:0/3190605920 wait complete. 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 Processor -- start 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 -- start start 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b4c1003a0 0x7f6b4c195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c105470 0x7f6b4c195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c1964f0 0x7f6b4c1a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1a5710 con 0x7f6b4c105470 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1a5880 con 0x7f6b4c1003a0 2024-02-05T08:10:16.590 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b52385640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1a5a40 con 0x7f6b4c1964f0 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.587+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c105470 0x7f6b4c195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c1964f0 0x7f6b4c1a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b4b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c105470 0x7f6b4c195fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46802/0 (socket says 172.21.15.43:46802) 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c1964f0 0x7f6b4c1a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:43408/0 (socket says 172.21.15.43:43408) 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b4b7fe640 1 -- 172.21.15.43:0/4010043172 learned_addr learned my addr 172.21.15.43:0/4010043172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b4bfff640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b4c1003a0 0x7f6b4c195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 -- 172.21.15.43:0/4010043172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b4c1003a0 msgr2=0x7f6b4c195a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b4c1003a0 0x7f6b4c195a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 -- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c105470 msgr2=0x7f6b4c195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c105470 0x7f6b4c195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b44009900 con 0x7f6b4c1964f0 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b4b7fe640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c105470 0x7f6b4c195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b50922640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c1964f0 0x7f6b4c1a50f0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400efc0 tx=0x7f6b4400c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b44003970 con 0x7f6b4c1964f0 2024-02-05T08:10:16.591 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3c009490 con 0x7f6b4c1964f0 2024-02-05T08:10:16.592 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6b44004020 con 0x7f6b4c1964f0 2024-02-05T08:10:16.592 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b4400b870 con 0x7f6b4c1964f0 2024-02-05T08:10:16.592 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.588+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c1a60a0 con 0x7f6b4c1964f0 2024-02-05T08:10:16.592 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.589+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c076fd0 con 0x7f6b4c1964f0 2024-02-05T08:10:16.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.590+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f6b4400a040 con 0x7f6b4c1964f0 2024-02-05T08:10:16.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.590+0000 7f6b497fa640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b3406da60 0x7f6b3406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:16.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.590+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(793..793 src has 257..793) v4 ==== 9455+0+0 (secure 0 0 0) 0x7f6b4408e2a0 con 0x7f6b4c1964f0 2024-02-05T08:10:16.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.590+0000 7f6b4bfff640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b3406da60 0x7f6b3406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:16.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.590+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=794}) v3 -- 0x7f6b34079d10 con 0x7f6b4c1964f0 2024-02-05T08:10:16.593 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.591+0000 7f6b4bfff640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b3406da60 0x7f6b3406ff20 secure :-1 s=READY pgs=4096 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009860 tx=0x7f6b3c0058b0 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:10:16.595 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.595+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b44056ed0 con 0x7f6b4c1964f0 2024-02-05T08:10:16.777 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:16.777+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b4c04f070 con 0x7f6b4c1964f0 2024-02-05T08:10:16.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:16 smithi043 ceph-mon[29879]: pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:16.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:16 smithi043 ceph-mon[41444]: pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:16.821 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:16 smithi050 ceph-mon[33492]: pgmap v1790: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-05T08:10:17.444 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.444+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(794..794 src has 257..794) v4 ==== 296+0+0 (secure 0 0 0) 0x7f6b4405b330 con 0x7f6b4c1964f0 2024-02-05T08:10:17.444 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.444+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=795}) v3 -- 0x7f6b3407a5a0 con 0x7f6b4c1964f0 2024-02-05T08:10:17.447 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.447+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]=0 pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' removed v794) v1 ==== 248+0+0 (secure 0 0 0) 0x7f6b4405a590 con 0x7f6b4c1964f0 2024-02-05T08:10:17.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.564+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b4c100f20 con 0x7f6b4c1964f0 2024-02-05T08:10:17.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.564+0000 7f6b497fa640 1 -- 172.21.15.43:0/4010043172 <== mon.2 v2:172.21.15.43:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]=0 pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' does not exist v794) v1 ==== 255+0+0 (secure 0 0 0) 0x7f6b4405a770 con 0x7f6b4c1964f0 2024-02-05T08:10:17.565 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'e2a6ede4-caf1-4034-97c2-ddac1835f21e' does not exist 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b3406da60 msgr2=0x7f6b3406ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b3406da60 0x7f6b3406ff20 secure :-1 s=READY pgs=4096 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009860 tx=0x7f6b3c0058b0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c1964f0 msgr2=0x7f6b4c1a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c1964f0 0x7f6b4c1a50f0 secure :-1 s=READY pgs=3201 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400efc0 tx=0x7f6b4400c800 comp rx=0 tx=0).stop 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 shutdown_connections 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f6b3406da60 0x7f6b3406ff20 unknown :-1 s=CLOSED pgs=4096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f6b4c1964f0 0x7f6b4c1a50f0 unknown :-1 s=CLOSED pgs=3201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6b4c105470 0x7f6b4c195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.566+0000 7f6b52385640 1 --2- 172.21.15.43:0/4010043172 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f6b4c1003a0 0x7f6b4c195a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.567+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 >> 172.21.15.43:0/4010043172 conn(0x7f6b4c0764b0 msgr2=0x7f6b4c0fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.567+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 shutdown_connections 2024-02-05T08:10:17.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.567+0000 7f6b52385640 1 -- 172.21.15.43:0/4010043172 wait complete. 2024-02-05T08:10:17.576 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 f0d5bf71-c3d4-4701-ac36-dcd79d7c0864 --yes-i-really-really-mean-it 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 -- 172.21.15.43:0/635962067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a4100d50 msgr2=0x7f37a4104d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 --2- 172.21.15.43:0/635962067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a4100d50 0x7f37a4104d90 secure :-1 s=READY pgs=3035 cs=0 l=1 rev1=1 crypto rx=0x7f3798009930 tx=0x7f379802bc30 comp rx=0 tx=0).stop 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 -- 172.21.15.43:0/635962067 shutdown_connections 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 --2- 172.21.15.43:0/635962067 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a4105470 0x7f37a410d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 --2- 172.21.15.43:0/635962067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a4100d50 0x7f37a4104d90 unknown :-1 s=CLOSED pgs=3035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 --2- 172.21.15.43:0/635962067 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a41003a0 0x7f37a4100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 -- 172.21.15.43:0/635962067 >> 172.21.15.43:0/635962067 conn(0x7f37a4077260 msgr2=0x7f37a4077670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 -- 172.21.15.43:0/635962067 shutdown_connections 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.660+0000 7f37aa380640 1 -- 172.21.15.43:0/635962067 wait complete. 2024-02-05T08:10:17.661 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 Processor -- start 2024-02-05T08:10:17.662 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 -- start start 2024-02-05T08:10:17.663 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a41003a0 0x7f37a4196e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a4105470 0x7f37a4197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 --2- >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a419b8f0 0x7f37a419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a419c2e0 con 0x7f37a41003a0 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 -- --> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a419c450 con 0x7f37a419b8f0 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37aa380640 1 -- --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a419c5c0 con 0x7f37a4105470 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a41003a0 0x7f37a4196e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37a37fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a4105470 0x7f37a4197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37a37fe640 1 --2- >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a4105470 0x7f37a4197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3301/0 says I am v2:172.21.15.43:43426/0 (socket says 172.21.15.43:43426) 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37a3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a41003a0 0x7f37a4196e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:46830/0 (socket says 172.21.15.43:46830) 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37a37fe640 1 -- 172.21.15.43:0/3734273087 learned_addr learned my addr 172.21.15.43:0/3734273087 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.661+0000 7f37a891d640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a419b8f0 0x7f37a419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:17.664 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a37fe640 1 -- 172.21.15.43:0/3734273087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a419b8f0 msgr2=0x7f37a419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a37fe640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a419b8f0 0x7f37a419bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a37fe640 1 -- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a41003a0 msgr2=0x7f37a4196e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a37fe640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a41003a0 0x7f37a4196e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a37fe640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798009560 con 0x7f37a4105470 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a3fff640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a41003a0 0x7f37a4196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a37fe640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a4105470 0x7f37a4197360 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7f3798009230 tx=0x7f3798004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37980035f0 con 0x7f37a4105470 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a41a3ba0 con 0x7f37a4105470 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3798003bd0 con 0x7f37a4105470 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37980054e0 con 0x7f37a4105470 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.662+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a41a4050 con 0x7f37a4105470 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.663+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a4101870 con 0x7f37a4105470 2024-02-05T08:10:17.665 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.664+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 4 ==== mgrmap(e 22) v1 ==== 91463+0+0 (secure 0 0 0) 0x7f3798003750 con 0x7f37a4105470 2024-02-05T08:10:17.666 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.664+0000 7f37a17fa640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f378c06da30 0x7f378c06fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-05T08:10:17.666 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.664+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 5 ==== osd_map(794..794 src has 257..794) v4 ==== 9067+0+0 (secure 0 0 0) 0x7f37980b0450 con 0x7f37a4105470 2024-02-05T08:10:17.666 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.664+0000 7f37a3fff640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f378c06da30 0x7f378c06fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-05T08:10:17.666 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.664+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=795}) v3 -- 0x7f378c0794c0 con 0x7f37a4105470 2024-02-05T08:10:17.666 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.664+0000 7f37a3fff640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f378c06da30 0x7f378c06fef0 secure :-1 s=READY pgs=4097 cs=0 l=1 rev1=1 crypto rx=0x7f3794009fc0 tx=0x7f3794005f90 comp rx=0 tx=0).ready entity=mgr.24485 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-05T08:10:17.668 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.669+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f37980792b0 con 0x7f37a4105470 2024-02-05T08:10:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:17 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4010043172' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:17.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:17 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:17.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:17 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4010043172' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:17.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:17 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:17 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4010043172' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:17.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:17 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:17.853 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:17.853+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37a404f070 con 0x7f37a4105470 2024-02-05T08:10:18.446 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.445+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 7 ==== osd_map(795..795 src has 257..795) v4 ==== 296+0+0 (secure 0 0 0) 0x7f3798003d40 con 0x7f37a4105470 2024-02-05T08:10:18.446 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.446+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_subscribe({osdmap=796}) v3 -- 0x7f378c079d10 con 0x7f37a4105470 2024-02-05T08:10:18.450 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.450+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]=0 pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' removed v795) v1 ==== 248+0+0 (secure 0 0 0) 0x7f3798043020 con 0x7f37a4105470 2024-02-05T08:10:18.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.569+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 --> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f37a41108a0 con 0x7f37a4105470 2024-02-05T08:10:18.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.569+0000 7f37a17fa640 1 -- 172.21.15.43:0/3734273087 <== mon.2 v2:172.21.15.43:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]=0 pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' does not exist v795) v1 ==== 255+0+0 (secure 0 0 0) 0x7f379807c970 con 0x7f37a4105470 2024-02-05T08:10:18.570 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'f0d5bf71-c3d4-4701-ac36-dcd79d7c0864' does not exist 2024-02-05T08:10:18.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f378c06da30 msgr2=0x7f378c06fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:18.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f378c06da30 0x7f378c06fef0 secure :-1 s=READY pgs=4097 cs=0 l=1 rev1=1 crypto rx=0x7f3794009fc0 tx=0x7f3794005f90 comp rx=0 tx=0).stop 2024-02-05T08:10:18.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a4105470 msgr2=0x7f37a4197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-05T08:10:18.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a4105470 0x7f37a4197360 secure :-1 s=READY pgs=3202 cs=0 l=1 rev1=1 crypto rx=0x7f3798009230 tx=0x7f3798004270 comp rx=0 tx=0).stop 2024-02-05T08:10:18.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 shutdown_connections 2024-02-05T08:10:18.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:6800/847956893,v1:172.21.15.43:6801/847956893] conn(0x7f378c06da30 0x7f378c06fef0 unknown :-1 s=CLOSED pgs=4097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:18.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] conn(0x7f37a419b8f0 0x7f37a419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:18.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3301/0,v1:172.21.15.43:6790/0] conn(0x7f37a4105470 0x7f37a4197360 unknown :-1 s=CLOSED pgs=3202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:18.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 --2- 172.21.15.43:0/3734273087 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37a41003a0 0x7f37a4196e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-05T08:10:18.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 >> 172.21.15.43:0/3734273087 conn(0x7f37a4077260 msgr2=0x7f37a40fdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-05T08:10:18.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 shutdown_connections 2024-02-05T08:10:18.573 INFO:tasks.workunit.client.0.smithi043.stderr:2024-02-05T08:10:18.572+0000 7f37aa380640 1 -- 172.21.15.43:0/3734273087 wait complete. 2024-02-05T08:10:18.581 INFO:tasks.workunit.client.0.smithi043.stdout:OK 2024-02-05T08:10:18.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo OK 2024-02-05T08:10:18.583 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-05T08:10:18.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-05T08:10:18.652 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-05T08:10:18.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-05T08:10:18.715 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[29879]: pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[29879]: osdmap e794: 8 total, 8 up, 8 in 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/4010043172' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3734273087' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[41444]: pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:10:18.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[41444]: osdmap e794: 8 total, 8 up, 8 in 2024-02-05T08:10:18.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/4010043172' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3734273087' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:18 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:18 smithi050 ceph-mon[33492]: pgmap v1791: 188 pgs: 188 active+clean; 863 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-05T08:10:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:10:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:18 smithi050 ceph-mon[33492]: osdmap e794: 8 total, 8 up, 8 in 2024-02-05T08:10:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/4010043172' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "pool2": "e2a6ede4-caf1-4034-97c2-ddac1835f21e", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:18 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3734273087' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.822 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:18 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:18.963 DEBUG:teuthology.parallel:result is None 2024-02-05T08:10:18.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-05T08:10:19.030 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-05T08:10:19.030 DEBUG:teuthology.orchestra.run.smithi043:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-05T08:10:19.126 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-05T08:10:19.126 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-05T08:10:19.139 INFO:tasks.cephadm:Teardown begin 2024-02-05T08:10:19.139 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:10:19.166 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:10:19.201 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-05T08:10:19.201 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-05T08:10:19.221 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-05T08:10:19.259 INFO:tasks.cephadm:Stopping all daemons... 2024-02-05T08:10:19.259 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-05T08:10:19.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a 2024-02-05T08:10:19.284 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: ::ffff:172.21.15.50 - - [05/Feb/2024:08:10:19] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-05T08:10:19.549 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 systemd[1]: Stopping Ceph mon.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:19.549 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[29879]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:10:19.549 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:10:19.549 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[29879]: osdmap e795: 8 total, 8 up, 8 in 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[29879]: from='client.? 172.21.15.43:0/3734273087' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[29879]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[29879]: pgmap v1794: 164 pgs: 164 active+clean; 582 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[41444]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[41444]: osdmap e795: 8 total, 8 up, 8 in 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[41444]: from='client.? 172.21.15.43:0/3734273087' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[41444]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:19.550 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-mon[41444]: pgmap v1794: 164 pgs: 164 active+clean; 582 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:10:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a[29856]: 2024-02-05T08:10:19.548+0000 7f64ec2bc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a[29856]: 2024-02-05T08:10:19.548+0000 7f64ec2bc700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-05T08:10:19.812 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:19 smithi043 podman[160201]: 2024-02-05 08:10:19.620725991 +0000 UTC m=+0.175466907 container died cd9e9007486b18bb0d10503fc893f0f528fc49aa015a9df29590f10def18d65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-05T08:10:19.823 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:19 smithi050 ceph-mon[33492]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-05T08:10:19.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]': finished 2024-02-05T08:10:19.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:19 smithi050 ceph-mon[33492]: osdmap e795: 8 total, 8 up, 8 in 2024-02-05T08:10:19.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:19 smithi050 ceph-mon[33492]: from='client.? 172.21.15.43:0/3734273087' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:19.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:19 smithi050 ceph-mon[33492]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "pool2": "f0d5bf71-c3d4-4701-ac36-dcd79d7c0864", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-05T08:10:19.824 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:19 smithi050 ceph-mon[33492]: pgmap v1794: 164 pgs: 164 active+clean; 582 KiB data, 846 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-05T08:10:20.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:20 smithi043 podman[160201]: 2024-02-05 08:10:20.054700932 +0000 UTC m=+0.609441849 container cleanup cd9e9007486b18bb0d10503fc893f0f528fc49aa015a9df29590f10def18d65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, ceph=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-05T08:10:20.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:20 smithi043 bash[160201]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a 2024-02-05T08:10:20.479 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service' 2024-02-05T08:10:20.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:20 smithi043 podman[160219]: 2024-02-05 08:10:20.264988573 +0000 UTC m=+0.640866071 container remove cd9e9007486b18bb0d10503fc893f0f528fc49aa015a9df29590f10def18d65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:10:20.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:20 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-05T08:10:20.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:20 smithi043 systemd[1]: Stopped Ceph mon.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:10:20.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Feb 05 08:10:20 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.a.service: Consumed 2min 18.097s CPU time. 2024-02-05T08:10:20.559 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:20.559 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-05T08:10:20.559 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-05T08:10:20.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.c 2024-02-05T08:10:20.812 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:20 smithi043 systemd[1]: Stopping Ceph mon.c for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:20 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c[41420]: 2024-02-05T08:10:20.911+0000 7fdfee0d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:20 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c[41420]: 2024-02-05T08:10:20.911+0000 7fdfee0d4700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:20 smithi043 podman[160334]: 2024-02-05 08:10:20.954141697 +0000 UTC m=+0.175268957 container died 223c6dab2a8206070db1eabae34b926e5691c570d168e8eab4f35cbd17d9d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:20 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:08:10:20] ENGINE Bus STOPPING 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:20 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:08:10:20] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:20 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:08:10:20] ENGINE Bus STOPPED 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:20 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:08:10:20] ENGINE Bus STARTING 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:08:10:21] ENGINE Serving on http://:::9283 2024-02-05T08:10:21.219 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:21 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: [05/Feb/2024:08:10:21] ENGINE Bus STARTED 2024-02-05T08:10:21.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:21 smithi043 podman[160334]: 2024-02-05 08:10:21.310025437 +0000 UTC m=+0.531152697 container cleanup 223c6dab2a8206070db1eabae34b926e5691c570d168e8eab4f35cbd17d9d2c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-05T08:10:21.562 INFO:journalctl@ceph.mon.c.smithi043.stdout:Feb 05 08:10:21 smithi043 bash[160334]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-c 2024-02-05T08:10:21.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.c.service' 2024-02-05T08:10:21.817 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:21.817 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-05T08:10:21.817 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-05T08:10:21.817 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.b 2024-02-05T08:10:22.132 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:21 smithi050 systemd[1]: Stopping Ceph mon.b for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:22.441 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b[33469]: 2024-02-05T08:10:22.130+0000 7fe39d261700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:22.441 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b[33469]: 2024-02-05T08:10:22.130+0000 7fe39d261700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-05T08:10:22.441 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:22 smithi050 podman[79140]: 2024-02-05 08:10:22.203682198 +0000 UTC m=+0.188791990 container died 18b270fd5e95df227e43d5f6ca9d9359caa5d46c2a1f9ce92b1df4d24dee26b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-05T08:10:22.721 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:10:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug there is no tcmu-runner data available 2024-02-05T08:10:22.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:22 smithi050 podman[79140]: 2024-02-05 08:10:22.721873499 +0000 UTC m=+0.706983325 container cleanup 18b270fd5e95df227e43d5f6ca9d9359caa5d46c2a1f9ce92b1df4d24dee26b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-05T08:10:22.973 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:22 smithi050 bash[79140]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b 2024-02-05T08:10:23.197 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.b.service' 2024-02-05T08:10:23.223 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:22 smithi050 podman[79159]: 2024-02-05 08:10:22.973374305 +0000 UTC m=+0.768784526 container remove 18b270fd5e95df227e43d5f6ca9d9359caa5d46c2a1f9ce92b1df4d24dee26b7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mon-b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:23.223 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:23 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-05T08:10:23.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:23 smithi050 systemd[1]: Stopped Ceph mon.b for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:10:23.224 INFO:journalctl@ceph.mon.b.smithi050.stdout:Feb 05 08:10:23 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mon.b.service: Consumed 33.173s CPU time. 2024-02-05T08:10:23.243 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:23.243 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-05T08:10:23.243 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-05T08:10:23.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y 2024-02-05T08:10:23.558 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:23 smithi043 systemd[1]: Stopping Ceph mgr.y for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:23.812 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T08:10:23.558+0000 7f0e894a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:23.812 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:23 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y[30136]: 2024-02-05T08:10:23.558+0000 7f0e894a4700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-05T08:10:23.812 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:23 smithi043 podman[160477]: 2024-02-05 08:10:23.590084593 +0000 UTC m=+0.157989265 container died 240db77c263ddc893bc4550df38b329f8ca6731d256f8a43729e24c525edb8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:24.300 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:24 smithi043 podman[160477]: 2024-02-05 08:10:24.115143275 +0000 UTC m=+0.683047946 container cleanup 240db77c263ddc893bc4550df38b329f8ca6731d256f8a43729e24c525edb8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-05T08:10:24.300 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:24 smithi043 bash[160477]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y 2024-02-05T08:10:24.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y.service' 2024-02-05T08:10:24.556 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:24 smithi043 podman[160495]: 2024-02-05 08:10:24.300525986 +0000 UTC m=+0.709571232 container remove 240db77c263ddc893bc4550df38b329f8ca6731d256f8a43729e24c525edb8c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:164d3be4a527be775f4d3c343ea591122f1b8921, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-y, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-05T08:10:24.556 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:24 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y.service: Deactivated successfully. 2024-02-05T08:10:24.556 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:24 smithi043 systemd[1]: Stopped Ceph mgr.y for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:10:24.556 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Feb 05 08:10:24 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.y.service: Consumed 2min 10.202s CPU time. 2024-02-05T08:10:24.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:24.569 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-05T08:10:24.569 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-05T08:10:24.569 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.x 2024-02-05T08:10:24.900 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:24 smithi050 systemd[1]: Stopping Ceph mgr.x for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:25.322 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:24 smithi050 podman[79271]: 2024-02-05 08:10:24.900426166 +0000 UTC m=+0.133071936 container stop 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-05T08:10:25.322 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:24 smithi050 podman[79271]: 2024-02-05 08:10:24.927586406 +0000 UTC m=+0.160232187 container died 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_CLEAN=True) 2024-02-05T08:10:25.627 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:25 smithi050 podman[79271]: 2024-02-05 08:10:25.443525269 +0000 UTC m=+0.676171038 container cleanup 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-05T08:10:25.628 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:25 smithi050 bash[79271]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x 2024-02-05T08:10:25.868 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.x.service' 2024-02-05T08:10:25.893 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:25 smithi050 podman[79290]: 2024-02-05 08:10:25.628317126 +0000 UTC m=+0.722833376 container remove 730d692666e18c7311cc9c087c7372647b2f78e59d05339d471b1776f9cb7e12 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-mgr-x, io.buildah.version=1.33.2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=) 2024-02-05T08:10:25.893 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:25 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:10:25.893 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:25 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-02-05T08:10:25.893 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:25 smithi050 systemd[1]: Stopped Ceph mgr.x for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:10:25.893 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Feb 05 08:10:25 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@mgr.x.service: Consumed 35.754s CPU time. 2024-02-05T08:10:25.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:25.905 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-05T08:10:25.906 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-05T08:10:25.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.0 2024-02-05T08:10:26.213 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:25 smithi043 systemd[1]: Stopping Ceph osd.0 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:26.562 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:26 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0[45879]: 2024-02-05T08:10:26.213+0000 7f82d6f64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:26.562 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:26 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0[45879]: 2024-02-05T08:10:26.213+0000 7f82d6f64700 -1 osd.0 795 *** Got signal Terminated *** 2024-02-05T08:10:26.562 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:26 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0[45879]: 2024-02-05T08:10:26.213+0000 7f82d6f64700 -1 osd.0 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:10:32.012 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:31 smithi043 podman[160608]: 2024-02-05 08:10:31.746031315 +0000 UTC m=+5.651269025 container died 367e69aff4fefebb9587a7dc1b0b692d7faf054fd591b16cbb38fdbe1cd1ab88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-02-05T08:10:32.312 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:32 smithi043 podman[160608]: 2024-02-05 08:10:32.102887395 +0000 UTC m=+6.008125156 container cleanup 367e69aff4fefebb9587a7dc1b0b692d7faf054fd591b16cbb38fdbe1cd1ab88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-05T08:10:32.312 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:32 smithi043 bash[160608]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0 2024-02-05T08:10:32.716 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:32 smithi043 podman[160630]: 2024-02-05 08:10:32.37075551 +0000 UTC m=+0.625931075 container remove 367e69aff4fefebb9587a7dc1b0b692d7faf054fd591b16cbb38fdbe1cd1ab88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:10:33.040 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:32 smithi043 podman[160729]: 2024-02-05 08:10:32.620012406 +0000 UTC m=+0.018013817 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:10:33.040 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:32 smithi043 podman[160729]: 2024-02-05 08:10:32.746697142 +0000 UTC m=+0.144698588 container create 0b23b9533eb215f5f0ff9210fbd4096585b1a3ead704fc6be984a54d8d430aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-05T08:10:33.040 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:32 smithi043 podman[160729]: 2024-02-05 08:10:32.957956619 +0000 UTC m=+0.355958038 container init 0b23b9533eb215f5f0ff9210fbd4096585b1a3ead704fc6be984a54d8d430aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-05T08:10:33.040 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:32 smithi043 podman[160729]: 2024-02-05 08:10:32.963927411 +0000 UTC m=+0.361928817 container start 0b23b9533eb215f5f0ff9210fbd4096585b1a3ead704fc6be984a54d8d430aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:33.041 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:33 smithi043 podman[160729]: 2024-02-05 08:10:33.040759884 +0000 UTC m=+0.438761287 container attach 0b23b9533eb215f5f0ff9210fbd4096585b1a3ead704fc6be984a54d8d430aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-02-05T08:10:33.312 INFO:journalctl@ceph.osd.0.smithi043.stdout:Feb 05 08:10:33 smithi043 podman[160729]: 2024-02-05 08:10:33.259505666 +0000 UTC m=+0.657507069 container died 0b23b9533eb215f5f0ff9210fbd4096585b1a3ead704fc6be984a54d8d430aa4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS) 2024-02-05T08:10:33.987 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.0.service' 2024-02-05T08:10:34.030 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:34.030 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-05T08:10:34.031 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-05T08:10:34.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.1 2024-02-05T08:10:34.312 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:34 smithi043 systemd[1]: Stopping Ceph osd.1 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:34.812 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1[52148]: 2024-02-05T08:10:34.366+0000 7f7e1b5c5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:34.812 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1[52148]: 2024-02-05T08:10:34.366+0000 7f7e1b5c5700 -1 osd.1 795 *** Got signal Terminated *** 2024-02-05T08:10:34.812 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:34 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1[52148]: 2024-02-05T08:10:34.366+0000 7f7e1b5c5700 -1 osd.1 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:10:40.219 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:39 smithi043 podman[160860]: 2024-02-05 08:10:39.936152242 +0000 UTC m=+5.674698175 container died 08b8b10846a8049397901a6e9061ee242b0a97f8bfb4beb012aeea6dcf0a4215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-05T08:10:40.540 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:40 smithi043 podman[160860]: 2024-02-05 08:10:40.314229117 +0000 UTC m=+6.052775048 container cleanup 08b8b10846a8049397901a6e9061ee242b0a97f8bfb4beb012aeea6dcf0a4215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:40.540 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:40 smithi043 bash[160860]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1 2024-02-05T08:10:40.812 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:40 smithi043 podman[160882]: 2024-02-05 08:10:40.54043905 +0000 UTC m=+0.605369071 container remove 08b8b10846a8049397901a6e9061ee242b0a97f8bfb4beb012aeea6dcf0a4215 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-05T08:10:41.143 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:40 smithi043 podman[160979]: 2024-02-05 08:10:40.754178385 +0000 UTC m=+0.018543243 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:10:41.143 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:40 smithi043 podman[160979]: 2024-02-05 08:10:40.858020102 +0000 UTC m=+0.122384960 container create 81051392032dc152b338af7f3cb23b89933a811e652c9f3dede02a8ce6a2dab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2) 2024-02-05T08:10:41.143 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:41 smithi043 podman[160979]: 2024-02-05 08:10:41.052039697 +0000 UTC m=+0.316404548 container init 81051392032dc152b338af7f3cb23b89933a811e652c9f3dede02a8ce6a2dab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-05T08:10:41.143 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:41 smithi043 podman[160979]: 2024-02-05 08:10:41.057836862 +0000 UTC m=+0.322201720 container start 81051392032dc152b338af7f3cb23b89933a811e652c9f3dede02a8ce6a2dab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:10:41.143 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:41 smithi043 podman[160979]: 2024-02-05 08:10:41.143478532 +0000 UTC m=+0.407843395 container attach 81051392032dc152b338af7f3cb23b89933a811e652c9f3dede02a8ce6a2dab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-05T08:10:41.562 INFO:journalctl@ceph.osd.1.smithi043.stdout:Feb 05 08:10:41 smithi043 podman[160979]: 2024-02-05 08:10:41.357558646 +0000 UTC m=+0.621923505 container died 81051392032dc152b338af7f3cb23b89933a811e652c9f3dede02a8ce6a2dab6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:10:41.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.1.service' 2024-02-05T08:10:42.025 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:42.025 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-05T08:10:42.025 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-05T08:10:42.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.2 2024-02-05T08:10:42.306 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:42 smithi043 systemd[1]: Stopping Ceph osd.2 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:42.562 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2[58158]: 2024-02-05T08:10:42.355+0000 7f2078cf7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:42.562 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2[58158]: 2024-02-05T08:10:42.355+0000 7f2078cf7700 -1 osd.2 795 *** Got signal Terminated *** 2024-02-05T08:10:42.562 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:42 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2[58158]: 2024-02-05T08:10:42.355+0000 7f2078cf7700 -1 osd.2 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:10:48.117 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:47 smithi043 podman[161112]: 2024-02-05 08:10:47.858118575 +0000 UTC m=+5.621497536 container died 87227ded1359cfa51e52752ae9ff8667d02e6638225795ddd6d7ffac18ccef97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS) 2024-02-05T08:10:48.525 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:48 smithi043 podman[161112]: 2024-02-05 08:10:48.208048132 +0000 UTC m=+5.971427094 container cleanup 87227ded1359cfa51e52752ae9ff8667d02e6638225795ddd6d7ffac18ccef97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux ) 2024-02-05T08:10:48.526 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:48 smithi043 bash[161112]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2 2024-02-05T08:10:48.812 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:48 smithi043 podman[161145]: 2024-02-05 08:10:48.526292665 +0000 UTC m=+0.670865494 container remove 87227ded1359cfa51e52752ae9ff8667d02e6638225795ddd6d7ffac18ccef97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:10:49.162 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:48 smithi043 podman[161242]: 2024-02-05 08:10:48.775015204 +0000 UTC m=+0.017596288 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:10:49.162 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:48 smithi043 podman[161242]: 2024-02-05 08:10:48.885589222 +0000 UTC m=+0.128170300 container create b434e7bcec21631e30fecf063cf0d019008bf830939c35e073c68d43039484e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-05T08:10:49.162 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:49 smithi043 podman[161242]: 2024-02-05 08:10:49.07127598 +0000 UTC m=+0.313857059 container init b434e7bcec21631e30fecf063cf0d019008bf830939c35e073c68d43039484e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD) 2024-02-05T08:10:49.163 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:49 smithi043 podman[161242]: 2024-02-05 08:10:49.077564978 +0000 UTC m=+0.320146056 container start b434e7bcec21631e30fecf063cf0d019008bf830939c35e073c68d43039484e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:49.163 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:49 smithi043 podman[161242]: 2024-02-05 08:10:49.162958756 +0000 UTC m=+0.405539849 container attach b434e7bcec21631e30fecf063cf0d019008bf830939c35e073c68d43039484e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2) 2024-02-05T08:10:49.562 INFO:journalctl@ceph.osd.2.smithi043.stdout:Feb 05 08:10:49 smithi043 podman[161242]: 2024-02-05 08:10:49.373534569 +0000 UTC m=+0.616115658 container died b434e7bcec21631e30fecf063cf0d019008bf830939c35e073c68d43039484e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:49.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.2.service' 2024-02-05T08:10:49.984 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:49.984 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-05T08:10:49.985 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-05T08:10:49.985 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.3 2024-02-05T08:10:50.305 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:50 smithi043 systemd[1]: Stopping Ceph osd.3 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:50.562 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:50 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3[64796]: 2024-02-05T08:10:50.304+0000 7f498b07f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:50.562 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:50 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3[64796]: 2024-02-05T08:10:50.304+0000 7f498b07f700 -1 osd.3 795 *** Got signal Terminated *** 2024-02-05T08:10:50.562 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:50 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3[64796]: 2024-02-05T08:10:50.304+0000 7f498b07f700 -1 osd.3 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:10:52.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:52 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:52.395+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:10:53.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:53 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:53.442+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:10:54.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:54 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:54.058+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:10:54.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:10:54 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:10:54.267+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:10:54.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:54 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:54.441+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:10:55.062 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:54 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3[64796]: 2024-02-05T08:10:54.727+0000 7f4983672700 -1 osd.3 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.950805+0000 front 2024-02-05T08:10:29.950824+0000 (oldest deadline 2024-02-05T08:10:54.051027+0000) 2024-02-05T08:10:55.432 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:10:55 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:10:55.219+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:10:55.460 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:55 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:55.084+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:10:55.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:55 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:55.431+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:10:55.886 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:55 smithi043 podman[161373]: 2024-02-05 08:10:55.635456399 +0000 UTC m=+5.430404258 container died 4d7792002fe22c253bdd762bf134d561b1b59f562f256f92483346b5240a55dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-05T08:10:56.144 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:55 smithi043 podman[161373]: 2024-02-05 08:10:55.97687292 +0000 UTC m=+5.771820838 container cleanup 4d7792002fe22c253bdd762bf134d561b1b59f562f256f92483346b5240a55dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:10:56.144 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:55 smithi043 bash[161373]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3 2024-02-05T08:10:56.387 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:56 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:56.111+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:10:56.387 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:10:56 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:10:56.250+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:10:56.562 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:56 smithi043 podman[161397]: 2024-02-05 08:10:56.144602931 +0000 UTC m=+0.506670015 container remove 4d7792002fe22c253bdd762bf134d561b1b59f562f256f92483346b5240a55dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:10:56.658 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:56 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:56.386+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:10:56.864 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:56 smithi043 podman[161494]: 2024-02-05 08:10:56.578998567 +0000 UTC m=+0.102988435 container create d78200d2d5affb07289557baaf1ca64ae0cc9ecb9e6c093221b4ec39718b71ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:56.864 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:56 smithi043 podman[161494]: 2024-02-05 08:10:56.493341377 +0000 UTC m=+0.017331246 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:10:56.864 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:56 smithi043 podman[161494]: 2024-02-05 08:10:56.789751427 +0000 UTC m=+0.313741296 container init d78200d2d5affb07289557baaf1ca64ae0cc9ecb9e6c093221b4ec39718b71ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-05T08:10:56.864 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:56 smithi043 podman[161494]: 2024-02-05 08:10:56.795269057 +0000 UTC m=+0.319258926 container start d78200d2d5affb07289557baaf1ca64ae0cc9ecb9e6c093221b4ec39718b71ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3-deactivate, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2) 2024-02-05T08:10:56.864 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:56 smithi043 podman[161494]: 2024-02-05 08:10:56.864528546 +0000 UTC m=+0.388518414 container attach d78200d2d5affb07289557baaf1ca64ae0cc9ecb9e6c093221b4ec39718b71ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:57.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:10:56 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:10:56.657+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:10:57.312 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:57 smithi043 podman[161494]: 2024-02-05 08:10:57.092100206 +0000 UTC m=+0.616090074 container died d78200d2d5affb07289557baaf1ca64ae0cc9ecb9e6c093221b4ec39718b71ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-05T08:10:57.411 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:57 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:57.074+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:10:57.411 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:10:57 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:10:57.245+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:10:57.661 INFO:journalctl@ceph.osd.3.smithi043.stdout:Feb 05 08:10:57 smithi043 podman[161541]: 2024-02-05 08:10:57.651102091 +0000 UTC m=+0.554462064 container remove d78200d2d5affb07289557baaf1ca64ae0cc9ecb9e6c093221b4ec39718b71ce (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-05T08:10:57.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.3.service' 2024-02-05T08:10:57.707 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:10:57.707 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-05T08:10:57.708 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-05T08:10:57.708 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.4 2024-02-05T08:10:57.739 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:57 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:57.410+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:10:57.740 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:10:57 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:10:57.658+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:10:58.027 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:57 smithi050 systemd[1]: Stopping Ceph osd.4 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:10:58.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:10:58.215+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:10:58.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:58.025+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:10:58.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:58.039+0000 7fc2f3c4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:10:58.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:58.039+0000 7fc2f3c4d700 -1 osd.4 795 *** Got signal Terminated *** 2024-02-05T08:10:58.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:58.039+0000 7fc2f3c4d700 -1 osd.4 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:10:58.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:58.448+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:10:58.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:10:58.684+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:10:59.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:10:59 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:10:59.173+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:10:59.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:58 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:58.989+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:10:59.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:10:59 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:10:59.665+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:10:59.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:10:59 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:10:59.460+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:00.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:10:59 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:10:59.962+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:11:00.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:00.144+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:00.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:00.459+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:00.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:00 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:00.671+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:01.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:01.143+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:01.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:11:01.009+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:11:01.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:11:01.009+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.456939+0000 front 2024-02-05T08:10:38.457123+0000 (oldest deadline 2024-02-05T08:11:00.156834+0000) 2024-02-05T08:11:01.688 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:01.422+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:01.976 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:01.686+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:02.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:11:01.975+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:11:02.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:01 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:11:01.975+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.456939+0000 front 2024-02-05T08:10:38.457123+0000 (oldest deadline 2024-02-05T08:11:00.156834+0000) 2024-02-05T08:11:02.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:02.143+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:02.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:02.143+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:02.681 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:02.402+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:02.681 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:02.402+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:02.966 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:02.680+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:02.966 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:02.680+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:03.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:11:02.965+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.256150+0000 front 2024-02-05T08:10:30.256213+0000 (oldest deadline 2024-02-05T08:10:53.155986+0000) 2024-02-05T08:11:03.322 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:02 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4[39040]: 2024-02-05T08:11:02.965+0000 7fc2ec240700 -1 osd.4 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.456939+0000 front 2024-02-05T08:10:38.457123+0000 (oldest deadline 2024-02-05T08:11:00.156834+0000) 2024-02-05T08:11:03.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:03 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:03.134+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:03.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:03 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:03.134+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:03.639 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:03 smithi050 podman[79424]: 2024-02-05 08:11:03.377215711 +0000 UTC m=+5.457480768 container died 55d51f1b9feec8e5749cde48eda4665dc62273a219cbcbe43e4ee4ae1e48c9b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:11:03.639 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:03 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:03.353+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:03.639 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:03 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:03.353+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:04.072 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:03 smithi050 podman[79424]: 2024-02-05 08:11:03.77106859 +0000 UTC m=+5.851333645 container cleanup 55d51f1b9feec8e5749cde48eda4665dc62273a219cbcbe43e4ee4ae1e48c9b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:11:04.072 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:03 smithi050 bash[79424]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4 2024-02-05T08:11:04.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:03 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:03.637+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:04.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:03 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:03.637+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:04.396 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:04 smithi050 podman[79446]: 2024-02-05 08:11:04.13078988 +0000 UTC m=+0.751169175 container remove 55d51f1b9feec8e5749cde48eda4665dc62273a219cbcbe43e4ee4ae1e48c9b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-05T08:11:04.397 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:04.162+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:04.397 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:04.162+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:04.397 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:04.201Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-02-05T08:11:04.397 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:04.203Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-02-05T08:11:04.397 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:04.203Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-02-05T08:11:04.397 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:04.204Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-02-05T08:11:04.648 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:04.395+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:04.648 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:04.395+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:04.648 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:04 smithi050 podman[79544]: 2024-02-05 08:11:04.348329562 +0000 UTC m=+0.018364493 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:11:04.648 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:04 smithi050 podman[79544]: 2024-02-05 08:11:04.490285033 +0000 UTC m=+0.160320002 container create 3102886decfd48f271f74a64798c5be27a4fb7ede7232e79a627e3424250fc02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2) 2024-02-05T08:11:04.649 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:04.607+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:04.649 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:04 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:04.607+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:05.031 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:04 smithi050 podman[79544]: 2024-02-05 08:11:04.726594541 +0000 UTC m=+0.396629476 container init 3102886decfd48f271f74a64798c5be27a4fb7ede7232e79a627e3424250fc02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:11:05.031 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:04 smithi050 podman[79544]: 2024-02-05 08:11:04.732621736 +0000 UTC m=+0.402656662 container start 3102886decfd48f271f74a64798c5be27a4fb7ede7232e79a627e3424250fc02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-02-05T08:11:05.031 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:04 smithi050 podman[79544]: 2024-02-05 08:11:04.834494542 +0000 UTC m=+0.504529497 container attach 3102886decfd48f271f74a64798c5be27a4fb7ede7232e79a627e3424250fc02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-05T08:11:05.311 INFO:journalctl@ceph.osd.4.smithi050.stdout:Feb 05 08:11:05 smithi050 podman[79544]: 2024-02-05 08:11:05.032157766 +0000 UTC m=+0.702192693 container died 3102886decfd48f271f74a64798c5be27a4fb7ede7232e79a627e3424250fc02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-05T08:11:05.312 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:05 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:05.170+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:05.312 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:05 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:05.170+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:05.572 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:05 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:05.433+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:05.572 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:05 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:05.433+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:05.762 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.4.service' 2024-02-05T08:11:05.844 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:11:05.845 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-05T08:11:05.845 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-05T08:11:05.845 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.5 2024-02-05T08:11:05.908 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:05 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:05.592+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:05.909 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:05 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:05.592+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:06.218 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:06.139+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:06.219 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:06.139+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:06.219 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:05 smithi050 systemd[1]: Stopping Ceph osd.5 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:06.219 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:06.217+0000 7f6b17941700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:11:06.475 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:06.217+0000 7f6b17941700 -1 osd.5 795 *** Got signal Terminated *** 2024-02-05T08:11:06.475 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:06.217+0000 7f6b17941700 -1 osd.5 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:11:06.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:06.474+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:06.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:06.474+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:06.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:06.578+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:06.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:06 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:06.578+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:07.507 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:07 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:07.150+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:07.507 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:07 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:07.150+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:07.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:07 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:07.505+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:07.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:07 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:07.505+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:07.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:07 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:07.558+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:07.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:07 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:07.558+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:08.461 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:08 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:08.193+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:08.461 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:08 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:08.193+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:08.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:08 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:08.460+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:08.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:08 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:08.460+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:08.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:08 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:08.569+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:08.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:08 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:08.569+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:09.457 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:09.154+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:09.457 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:09.154+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:09.457 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:09.154+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:09.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:09.543+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:09.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:09.543+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:09.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:09.456+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:09.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:09.456+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:09.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:09 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:09.456+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:43.735416+0000 front 2024-02-05T08:10:43.735435+0000 (oldest deadline 2024-02-05T08:11:09.035242+0000) 2024-02-05T08:11:10.482 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:10.199+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:10.482 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:10.199+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:10.482 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:10.199+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:10.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:10.539+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:10.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:10.539+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:10.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:10.482+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.632115+0000 front 2024-02-05T08:10:30.631930+0000 (oldest deadline 2024-02-05T08:10:52.332014+0000) 2024-02-05T08:11:10.823 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:10.482+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.334583+0000 front 2024-02-05T08:10:37.334647+0000 (oldest deadline 2024-02-05T08:11:01.434544+0000) 2024-02-05T08:11:10.823 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:10 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5[45371]: 2024-02-05T08:11:10.482+0000 7f6b0ff34700 -1 osd.5 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:43.735416+0000 front 2024-02-05T08:10:43.735435+0000 (oldest deadline 2024-02-05T08:11:09.035242+0000) 2024-02-05T08:11:11.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:11.243+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:11.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:11.243+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:11.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:11.243+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:12.072 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:11 smithi050 podman[79686]: 2024-02-05 08:11:11.766531203 +0000 UTC m=+5.680368180 container died b1e608e43e75f20410ad94658b3c8544796d4073219415c17d2a7c236caffd2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-05T08:11:12.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:11.574+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:12.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:11.574+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:12.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:11 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:11.574+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:12.492 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:12 smithi050 podman[79686]: 2024-02-05 08:11:12.249642169 +0000 UTC m=+6.163479136 container cleanup b1e608e43e75f20410ad94658b3c8544796d4073219415c17d2a7c236caffd2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2) 2024-02-05T08:11:12.495 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:12 smithi050 bash[79686]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5 2024-02-05T08:11:12.496 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:12.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:12.496 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:12.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:12.496 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:12.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:12.822 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:12 smithi050 podman[79709]: 2024-02-05 08:11:12.492544353 +0000 UTC m=+0.724832726 container remove b1e608e43e75f20410ad94658b3c8544796d4073219415c17d2a7c236caffd2e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-02-05T08:11:12.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:12.593+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:12.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:12.593+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:12.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:12 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:12.593+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:13.152 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:12 smithi050 podman[79804]: 2024-02-05 08:11:12.732592713 +0000 UTC m=+0.017982634 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:11:13.152 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:12 smithi050 podman[79804]: 2024-02-05 08:11:12.843658215 +0000 UTC m=+0.129048171 container create 7b0e4c53b5a7b880c386cc74f9df8dfd8697e5758532d2f895a3f0074b1470f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-05T08:11:13.152 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:13 smithi050 podman[79804]: 2024-02-05 08:11:13.071111481 +0000 UTC m=+0.356501395 container init 7b0e4c53b5a7b880c386cc74f9df8dfd8697e5758532d2f895a3f0074b1470f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-05T08:11:13.152 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:13 smithi050 podman[79804]: 2024-02-05 08:11:13.076636769 +0000 UTC m=+0.362026690 container start 7b0e4c53b5a7b880c386cc74f9df8dfd8697e5758532d2f895a3f0074b1470f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:11:13.572 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:13 smithi050 podman[79804]: 2024-02-05 08:11:13.15412697 +0000 UTC m=+0.439516901 container attach 7b0e4c53b5a7b880c386cc74f9df8dfd8697e5758532d2f895a3f0074b1470f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131) 2024-02-05T08:11:13.572 INFO:journalctl@ceph.osd.5.smithi050.stdout:Feb 05 08:11:13 smithi050 podman[79804]: 2024-02-05 08:11:13.366855084 +0000 UTC m=+0.652244998 container died 7b0e4c53b5a7b880c386cc74f9df8dfd8697e5758532d2f895a3f0074b1470f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:11:13.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:13 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:13.150+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:13.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:13 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:13.150+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:13.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:13 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:13.150+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:13.965 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:13 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:13.644+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:13.966 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:13 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:13.644+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:13.966 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:13 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:13.644+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:13.995 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.5.service' 2024-02-05T08:11:14.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:11:14.080 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-05T08:11:14.080 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-05T08:11:14.080 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.6 2024-02-05T08:11:14.322 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:14 smithi050 systemd[1]: Stopping Ceph osd.6 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:14.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:14.183+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:14.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:14.183+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:14.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:14.183+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:14.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:14.430+0000 7fb67952a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:11:14.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:14.430+0000 7fb67952a700 -1 osd.6 795 *** Got signal Terminated *** 2024-02-05T08:11:14.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:14.430+0000 7fb67952a700 -1 osd.6 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:11:14.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:14.666+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:14.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:14.666+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:14.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:14 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:14.666+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:15.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:15 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:15.202+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:15.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:15 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:15.202+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:15.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:15 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:15.202+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:16.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:15 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:15.625+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:16.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:15 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:15.625+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:16.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:15 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:15.625+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:16.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:16.154+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:16.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:16.154+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:16.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:16.154+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:16.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:16.154+0000 7f9282ae8700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24443.0:2192 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-05T08:11:17.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:16.601+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:17.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:16.601+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:17.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:16 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:16.601+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:17.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:17.126+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:17.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:17.126+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:17.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:17.127+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:17.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:17.127+0000 7f9282ae8700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24443.0:2192 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-05T08:11:18.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:17.587+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:18.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:17.587+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:18.072 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:17 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:17.587+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:18.557 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:18.172+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:18.557 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:18.172+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:18.557 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:18.172+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:18.557 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:18.172+0000 7f9282ae8700 -1 osd.7 795 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24443.0:2192 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e795) 2024-02-05T08:11:18.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:18.556+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:30.108210+0000 front 2024-02-05T08:10:30.108355+0000 (oldest deadline 2024-02-05T08:10:56.008061+0000) 2024-02-05T08:11:18.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:18.557+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:37.108900+0000 front 2024-02-05T08:10:37.109052+0000 (oldest deadline 2024-02-05T08:11:02.408938+0000) 2024-02-05T08:11:18.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:18.557+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:45.309954+0000 front 2024-02-05T08:10:45.310049+0000 (oldest deadline 2024-02-05T08:11:10.609915+0000) 2024-02-05T08:11:18.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:18 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6[51494]: 2024-02-05T08:11:18.557+0000 7fb671b1d700 -1 osd.6 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.111079+0000 front 2024-02-05T08:10:54.111046+0000 (oldest deadline 2024-02-05T08:11:17.610995+0000) 2024-02-05T08:11:19.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:19.173+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:19.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:19.173+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:19.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:19.173+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:19.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:19 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:19.173+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:20.195 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:19 smithi050 podman[79937]: 2024-02-05 08:11:19.917089228 +0000 UTC m=+5.618398267 container died e5132117ecf07c0a58f890d58e1ecb76976df807c39625e7c07c5879685aaa8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-05T08:11:20.544 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:20 smithi050 podman[79937]: 2024-02-05 08:11:20.293877314 +0000 UTC m=+5.995186350 container cleanup e5132117ecf07c0a58f890d58e1ecb76976df807c39625e7c07c5879685aaa8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=) 2024-02-05T08:11:20.545 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:20 smithi050 bash[79937]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6 2024-02-05T08:11:20.545 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:20.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:20.545 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:20.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:20.545 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:20.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:20.545 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:20 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:20.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:20.822 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:20 smithi050 podman[79961]: 2024-02-05 08:11:20.545140627 +0000 UTC m=+0.629573187 container remove e5132117ecf07c0a58f890d58e1ecb76976df807c39625e7c07c5879685aaa8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-02-05T08:11:21.116 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:20 smithi050 podman[80060]: 2024-02-05 08:11:20.752831489 +0000 UTC m=+0.018427503 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:11:21.116 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:20 smithi050 podman[80060]: 2024-02-05 08:11:20.854466654 +0000 UTC m=+0.120062707 container create b616b2bcb4d0071da319fb5fae1f8152041e062d327073cc7e90563644171b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:11:21.116 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:21 smithi050 podman[80060]: 2024-02-05 08:11:21.116299852 +0000 UTC m=+0.381895864 container init b616b2bcb4d0071da319fb5fae1f8152041e062d327073cc7e90563644171b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-05T08:11:21.418 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:21.190+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:21.418 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:21.190+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:21.418 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:21.190+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:21.418 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:21 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:21.190+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:21.419 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:21 smithi050 podman[80060]: 2024-02-05 08:11:21.122129068 +0000 UTC m=+0.387725081 container start b616b2bcb4d0071da319fb5fae1f8152041e062d327073cc7e90563644171b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:11:21.419 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:21 smithi050 podman[80060]: 2024-02-05 08:11:21.206926691 +0000 UTC m=+0.472522703 container attach b616b2bcb4d0071da319fb5fae1f8152041e062d327073cc7e90563644171b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-05T08:11:21.419 INFO:journalctl@ceph.osd.6.smithi050.stdout:Feb 05 08:11:21 smithi050 podman[80060]: 2024-02-05 08:11:21.418450705 +0000 UTC m=+0.684046725 container died b616b2bcb4d0071da319fb5fae1f8152041e062d327073cc7e90563644171b9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-6-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131) 2024-02-05T08:11:21.922 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.6.service' 2024-02-05T08:11:21.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:11:21.964 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-05T08:11:21.964 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-05T08:11:21.964 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.7 2024-02-05T08:11:22.316 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 systemd[1]: Stopping Ceph osd.7 for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:22.316 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:22.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:22.316 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:22.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:22.317 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:22.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:22.317 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:22.211+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:22.317 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:22.316+0000 7f92864ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:11:22.317 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:22.316+0000 7f92864ed700 -1 osd.7 795 *** Got signal Terminated *** 2024-02-05T08:11:22.317 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:22 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:22.316+0000 7f92864ed700 -1 osd.7 795 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-05T08:11:23.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:23.192+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:23.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:23.192+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:23.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:23.192+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:23.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:23 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:23.192+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:24.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:24 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:24.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:24.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:24 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:24.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:24.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:24 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:24.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:24.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:24 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:24.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:24.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:24 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:24.195+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.50:6804 osd.4 since back 2024-02-05T08:10:58.220500+0000 front 2024-02-05T08:10:58.220522+0000 (oldest deadline 2024-02-05T08:11:24.120507+0000) 2024-02-05T08:11:25.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:25 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:25.175+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:25.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:25 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:25.175+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:25.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:25 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:25.175+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:25.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:25 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:25.175+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:25.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:25 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:25.175+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.50:6804 osd.4 since back 2024-02-05T08:10:58.220500+0000 front 2024-02-05T08:10:58.220522+0000 (oldest deadline 2024-02-05T08:11:24.120507+0000) 2024-02-05T08:11:26.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:26 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:26.186+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:26.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:26 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:26.186+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:26.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:26 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:26.186+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:26.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:26 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:26.186+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:26.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:26 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:26.186+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.50:6804 osd.4 since back 2024-02-05T08:10:58.220500+0000 front 2024-02-05T08:10:58.220522+0000 (oldest deadline 2024-02-05T08:11:24.120507+0000) 2024-02-05T08:11:27.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:27 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:27.204+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6806 osd.0 since back 2024-02-05T08:10:29.816464+0000 front 2024-02-05T08:10:29.816522+0000 (oldest deadline 2024-02-05T08:10:53.916376+0000) 2024-02-05T08:11:27.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:27 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:27.204+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6814 osd.1 since back 2024-02-05T08:10:38.517790+0000 front 2024-02-05T08:10:38.517610+0000 (oldest deadline 2024-02-05T08:11:02.017674+0000) 2024-02-05T08:11:27.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:27 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:27.204+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6822 osd.2 since back 2024-02-05T08:10:46.619118+0000 front 2024-02-05T08:10:46.619094+0000 (oldest deadline 2024-02-05T08:11:08.219278+0000) 2024-02-05T08:11:27.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:27 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:27.204+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.43:6830 osd.3 since back 2024-02-05T08:10:54.120223+0000 front 2024-02-05T08:10:54.120286+0000 (oldest deadline 2024-02-05T08:11:18.220099+0000) 2024-02-05T08:11:27.572 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:27 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7[58022]: 2024-02-05T08:11:27.204+0000 7f9282ae8700 -1 osd.7 795 heartbeat_check: no reply from 172.21.15.50:6804 osd.4 since back 2024-02-05T08:10:58.220500+0000 front 2024-02-05T08:10:58.220522+0000 (oldest deadline 2024-02-05T08:11:24.120507+0000) 2024-02-05T08:11:28.030 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:27 smithi050 podman[80190]: 2024-02-05 08:11:27.750029888 +0000 UTC m=+5.558626659 container died 40f3699c47712ef90a0945c329e8c6c4891ff8aa3f283260141c362258c1270d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131) 2024-02-05T08:11:28.321 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:28 smithi050 podman[80190]: 2024-02-05 08:11:28.129308975 +0000 UTC m=+5.937905753 container cleanup 40f3699c47712ef90a0945c329e8c6c4891ff8aa3f283260141c362258c1270d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-05T08:11:28.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:28 smithi050 bash[80190]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7 2024-02-05T08:11:28.748 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:28 smithi050 podman[80212]: 2024-02-05 08:11:28.397253657 +0000 UTC m=+0.647594018 container remove 40f3699c47712ef90a0945c329e8c6c4891ff8aa3f283260141c362258c1270d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-05T08:11:29.016 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:28 smithi050 podman[80311]: 2024-02-05 08:11:28.651598283 +0000 UTC m=+0.017821500 image pull 8f4bf3e8b16aa617a3377544276cd5a5b73be8802746138ce876732ceeade8e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0 2024-02-05T08:11:29.016 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:28 smithi050 podman[80311]: 2024-02-05 08:11:28.756384904 +0000 UTC m=+0.122608159 container create 556cd9179c3ba4f82752165e3da01cab6eebe5d5ab08ab2f6dc7bcc73cb49656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-05T08:11:29.016 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:28 smithi050 podman[80311]: 2024-02-05 08:11:28.934916297 +0000 UTC m=+0.301139519 container init 556cd9179c3ba4f82752165e3da01cab6eebe5d5ab08ab2f6dc7bcc73cb49656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, org.label-schema.build-date=20240131) 2024-02-05T08:11:29.016 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:28 smithi050 podman[80311]: 2024-02-05 08:11:28.940807321 +0000 UTC m=+0.307030531 container start 556cd9179c3ba4f82752165e3da01cab6eebe5d5ab08ab2f6dc7bcc73cb49656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-05T08:11:29.016 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:29 smithi050 podman[80311]: 2024-02-05 08:11:29.016740372 +0000 UTC m=+0.382963595 container attach 556cd9179c3ba4f82752165e3da01cab6eebe5d5ab08ab2f6dc7bcc73cb49656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-05T08:11:29.322 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:29 smithi050 podman[80311]: 2024-02-05 08:11:29.234402347 +0000 UTC m=+0.600625560 container died 556cd9179c3ba4f82752165e3da01cab6eebe5d5ab08ab2f6dc7bcc73cb49656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131) 2024-02-05T08:11:29.772 INFO:journalctl@ceph.osd.7.smithi050.stdout:Feb 05 08:11:29 smithi050 podman[80357]: 2024-02-05 08:11:29.769794601 +0000 UTC m=+0.526639308 container remove 556cd9179c3ba4f82752165e3da01cab6eebe5d5ab08ab2f6dc7bcc73cb49656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131) 2024-02-05T08:11:29.799 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@osd.7.service' 2024-02-05T08:11:29.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:11:29.842 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-05T08:11:29.843 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-05T08:11:29.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@rgw.foo.a 2024-02-05T08:11:30.143 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 08:11:29 smithi043 systemd[1]: Stopping Ceph rgw.foo.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:30.418 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 08:11:30 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a[69769]: 2024-02-05T08:11:30.143+0000 7f8df9f6d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-05T08:11:30.418 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 08:11:30 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a[69769]: 2024-02-05T08:11:30.143+0000 7f8e087d8ac0 -1 shutting down 2024-02-05T08:11:40.466 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 08:11:40 smithi043 bash[161626]: time="2024-02-05T08:11:40Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-05T08:11:40.466 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 08:11:40 smithi043 podman[161626]: 2024-02-05 08:11:40.164370162 +0000 UTC m=+10.125387763 container died 1e8aad3b997601eff37eaa5153a2467d6fe539eabc1d333ed175afdad3bf278f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-02-05T08:11:40.812 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 08:11:40 smithi043 podman[161626]: 2024-02-05 08:11:40.620839477 +0000 UTC m=+10.581857084 container cleanup 1e8aad3b997601eff37eaa5153a2467d6fe539eabc1d333ed175afdad3bf278f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:11:40.812 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Feb 05 08:11:40 smithi043 bash[161626]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-rgw-foo-a 2024-02-05T08:11:41.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@rgw.foo.a.service' 2024-02-05T08:11:41.094 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:11:41.094 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-05T08:11:41.094 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-05T08:11:41.094 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@prometheus.a 2024-02-05T08:11:41.399 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 systemd[1]: Stopping Ceph prometheus.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:41.669 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-05T08:11:41.669 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.399Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.400Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.400Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a[68424]: ts=2024-02-05T08:11:41.400Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-05T08:11:41.670 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 podman[80442]: 2024-02-05 08:11:41.409809768 +0000 UTC m=+0.119572394 container died 5afd0d3c75b9f14961c9539226ac02f7a079bddabcbba404402d2bc09b291822 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:41.997 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 podman[80442]: 2024-02-05 08:11:41.813439251 +0000 UTC m=+0.523201874 container cleanup 5afd0d3c75b9f14961c9539226ac02f7a079bddabcbba404402d2bc09b291822 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:41.997 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Feb 05 08:11:41 smithi050 bash[80442]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-prometheus-a 2024-02-05T08:11:42.122 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@prometheus.a.service' 2024-02-05T08:11:42.161 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-05T08:11:42.162 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-05T08:11:42.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-05T08:11:42.353 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T08:11:44.062 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:43 smithi043 systemd[1]: Stopping Ceph alertmanager.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:44.479 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:44 smithi043 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a[80801]: ts=2024-02-05T08:11:44.197Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-05T08:11:44.479 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:44 smithi043 podman[162150]: 2024-02-05 08:11:44.239197599 +0000 UTC m=+0.148171330 container died 2c6ac989f290818a6b10833107a90a2b7c46511842bcc6514571a0b3b3c90bb3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:44.746 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:44 smithi043 podman[162150]: 2024-02-05 08:11:44.578827843 +0000 UTC m=+0.487801635 container cleanup 2c6ac989f290818a6b10833107a90a2b7c46511842bcc6514571a0b3b3c90bb3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:44.746 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:44 smithi043 bash[162150]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a 2024-02-05T08:11:45.033 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:44 smithi043 podman[162169]: 2024-02-05 08:11:44.746609709 +0000 UTC m=+0.503633426 container remove 2c6ac989f290818a6b10833107a90a2b7c46511842bcc6514571a0b3b3c90bb3 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:45.033 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:44 smithi043 podman[162169]: 2024-02-05 08:11:44.822322921 +0000 UTC m=+0.579346635 volume remove e20146e562f42b8bad4cad482897bab7ea0dc22d61e067b0d0d8f1e4f16c1490 2024-02-05T08:11:45.312 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:45 smithi043 systemd[1]: Stopping Ceph node-exporter.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:45.312 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:45 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-05T08:11:45.312 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:45 smithi043 systemd[1]: Stopped Ceph alertmanager.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:11:45.312 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Feb 05 08:11:45 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@alertmanager.a.service: Consumed 2.468s CPU time. 2024-02-05T08:11:45.766 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:45 smithi043 podman[162284]: 2024-02-05 08:11:45.513317107 +0000 UTC m=+0.119681727 container died 6d8340307e3272af692d04d68d973fe624ff5f60224c5d98dc1f18b73b4c5d59 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:46.062 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:45 smithi043 podman[162284]: 2024-02-05 08:11:45.866213555 +0000 UTC m=+0.472578178 container cleanup 6d8340307e3272af692d04d68d973fe624ff5f60224c5d98dc1f18b73b4c5d59 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:46.062 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:45 smithi043 bash[162284]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a 2024-02-05T08:11:46.439 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:46 smithi043 podman[162303]: 2024-02-05 08:11:46.142416321 +0000 UTC m=+0.630659179 container remove 6d8340307e3272af692d04d68d973fe624ff5f60224c5d98dc1f18b73b4c5d59 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-05T08:11:46.439 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:46 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:11:46.439 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:46 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-05T08:11:46.439 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:46 smithi043 systemd[1]: Stopped Ceph node-exporter.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:11:46.439 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Feb 05 08:11:46 smithi043 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.a.service: Consumed 5.330s CPU time. 2024-02-05T08:11:46.916 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-05T08:11:47.115 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T08:11:48.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:48 smithi050 systemd[1]: Stopping Ceph iscsi.iscsi.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:11:49.072 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:48 smithi050 ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a[63100]: debug Shutdown received 2024-02-05T08:11:58.917 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:58 smithi050 bash[80874]: time="2024-02-05T08:11:58Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-02-05T08:11:58.918 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:58 smithi050 podman[80874]: 2024-02-05 08:11:58.630423079 +0000 UTC m=+10.125295311 container died 8bf0cb917394ff82796509dec6a9da803341477e4de7810012b5bf93980d7895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:11:59.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:59 smithi050 podman[80874]: 2024-02-05 08:11:59.000145859 +0000 UTC m=+10.495018101 container cleanup 8bf0cb917394ff82796509dec6a9da803341477e4de7810012b5bf93980d7895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be) 2024-02-05T08:11:59.242 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:59 smithi050 bash[80874]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a 2024-02-05T08:11:59.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:59 smithi050 podman[80899]: 2024-02-05 08:11:59.242767403 +0000 UTC m=+0.614216338 container remove 8bf0cb917394ff82796509dec6a9da803341477e4de7810012b5bf93980d7895 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:471dc037e7bbfe27d19519404cf9282a911af42da176a0f19461b166165464b0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-iscsi-iscsi-a, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-3-2024-02-04-2254-164d3be, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-05T08:11:59.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:59 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-05T08:11:59.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:59 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-05T08:11:59.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:59 smithi050 systemd[1]: Stopped Ceph iscsi.iscsi.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:11:59.572 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Feb 05 08:11:59 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@iscsi.iscsi.a.service: Consumed 4.504s CPU time. 2024-02-05T08:12:01.213 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:00 smithi050 systemd[1]: Stopping Ceph grafana.a for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:12:01.466 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 podman[81178]: 2024-02-05 08:12:01.213819399 +0000 UTC m=+0.117677490 container died b2b6208640ac1a577734d837f99a3ff6c75a8e22a4b858963fa8fb415f49dc6c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, architecture=x86_64, description=Ceph Grafana Container, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., distribution-scope=public, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, version=8.5, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git) 2024-02-05T08:12:01.742 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 podman[81178]: 2024-02-05 08:12:01.550039575 +0000 UTC m=+0.453897665 container cleanup b2b6208640ac1a577734d837f99a3ff6c75a8e22a4b858963fa8fb415f49dc6c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, maintainer=Paul Cuzner , release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, vcs-type=git, description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, vendor=Red Hat, Inc., name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, distribution-scope=public) 2024-02-05T08:12:01.742 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 bash[81178]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a 2024-02-05T08:12:02.061 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 podman[81197]: 2024-02-05 08:12:01.742731109 +0000 UTC m=+0.530580258 container remove b2b6208640ac1a577734d837f99a3ff6c75a8e22a4b858963fa8fb415f49dc6c (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-grafana-a, distribution-scope=public, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.buildah.version=1.32.0, maintainer=Paul Cuzner , version=8.5, description=Ceph Grafana Container, io.openshift.expose-services=, release=239.1651231664, io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com) 2024-02-05T08:12:02.061 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:12:02.061 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-05T08:12:02.061 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 systemd[1]: Stopped Ceph grafana.a for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:12:02.061 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Feb 05 08:12:01 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@grafana.a.service: Consumed 18.313s CPU time. 2024-02-05T08:12:02.318 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:02 smithi050 systemd[1]: Stopping Ceph node-exporter.b for f211dd04-c3f7-11ee-95b6-87774f69a715... 2024-02-05T08:12:02.571 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:02 smithi050 podman[81315]: 2024-02-05 08:12:02.318272867 +0000 UTC m=+0.118675242 container died 536d1af5c6fd912559534524f7f4ef28ead1b5a4d7a9248475695c1f2db21978 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T08:12:02.929 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:02 smithi050 podman[81315]: 2024-02-05 08:12:02.73725639 +0000 UTC m=+0.537658800 container cleanup 536d1af5c6fd912559534524f7f4ef28ead1b5a4d7a9248475695c1f2db21978 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T08:12:02.929 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:02 smithi050 bash[81315]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b 2024-02-05T08:12:03.305 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:02 smithi050 podman[81333]: 2024-02-05 08:12:02.929807432 +0000 UTC m=+0.610554182 container remove 536d1af5c6fd912559534524f7f4ef28ead1b5a4d7a9248475695c1f2db21978 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f211dd04-c3f7-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-05T08:12:03.305 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:02 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-05T08:12:03.305 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:03 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-05T08:12:03.305 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:03 smithi050 systemd[1]: Stopped Ceph node-exporter.b for f211dd04-c3f7-11ee-95b6-87774f69a715. 2024-02-05T08:12:03.305 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Feb 05 08:12:03 smithi050 systemd[1]: ceph-f211dd04-c3f7-11ee-95b6-87774f69a715@node-exporter.b.service: Consumed 5.330s CPU time. 2024-02-05T08:12:03.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:12:03.812 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-05T08:12:03.841 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-05T08:12:03.843 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744/remote/smithi043/crash 2024-02-05T08:12:03.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/crash -- . 2024-02-05T08:12:03.896 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-05T08:12:03.897 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-02-05T08:12:03.899 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744/remote/smithi050/crash 2024-02-05T08:12:03.899 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/crash -- . 2024-02-05T08:12:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-05T08:12:03.933 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-02-05T08:12:03.934 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-05T08:12:03.934 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:12:03.967 INFO:teuthology.orchestra.run.smithi043.stdout:2024-02-05T07:40:00.000188+0000 mon.a (mon.0) 1944 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-05T08:12:03.968 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-05T08:12:03.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[SEC\]' /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:12:04.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]' /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:12:04.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[WRN\]' /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-02-05T08:12:04.183 INFO:teuthology.orchestra.run.smithi043.stdout:2024-02-05T07:40:00.000188+0000 mon.a (mon.0) 1944 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-02-05T08:12:04.184 INFO:tasks.cephadm:Compressing logs... 2024-02-05T08:12:04.184 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-05T08:12:04.226 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-05T08:12:04.250 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5: No such file or directory 2024-02-05T08:12:04.250 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.a.log 2024-02-05T08:12:04.250 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-05T08:12:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-05T08:12:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.1.log 2024-02-05T08:12:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.3.log 2024-02-05T08:12:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mgr.y.log 2024-02-05T08:12:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.0.log 2024-02-05T08:12:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.6.log 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:: No such file or directory 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.4.log 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.6.log: /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.7.log 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.5.log 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mgr.x.log 2024-02-05T08:12:04.258 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.7.log: /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/tcmu-runner.log 2024-02-05T08:12:04.259 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-05T08:12:04.259 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/tcmu-runner.log: 68.0%gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log 2024-02-05T08:12:04.259 INFO:teuthology.orchestra.run.smithi050.stderr: -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/tcmu-runner.log.gz 2024-02-05T08:12:04.259 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.cephadm.log: 81.7% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-05T08:12:04.259 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-05T08:12:04.259 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.b.log 2024-02-05T08:12:04.263 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-volume.log: 87.9% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-05T08:12:04.264 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-05T08:12:04.266 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.b.log: 92.3% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-05T08:12:04.269 INFO:teuthology.orchestra.run.smithi050.stderr: 92.6% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mgr.x.log.gz 2024-02-05T08:12:04.271 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.0.log: 95.3% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-05T08:12:04.272 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log 2024-02-05T08:12:04.274 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.cephadm.log: 89.0% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-05T08:12:04.274 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.2.log 2024-02-05T08:12:04.274 INFO:teuthology.orchestra.run.smithi050.stderr: 91.0% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-05T08:12:04.278 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/cephadm.log: 85.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-05T08:12:04.284 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log: 93.3% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-05T08:12:04.284 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-02-05T08:12:04.284 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.c.log 2024-02-05T08:12:04.292 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-05T08:12:04.299 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.c.log: 91.1% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-05T08:12:04.311 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/cephadm.log: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-05T08:12:04.325 INFO:teuthology.orchestra.run.smithi043.stderr: 94.3% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-05T08:12:04.828 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mgr.y.log.gz 2024-02-05T08:12:05.895 INFO:teuthology.orchestra.run.smithi050.stderr: 92.8% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.b.log.gz 2024-02-05T08:12:06.097 INFO:teuthology.orchestra.run.smithi043.stderr: 92.6% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.c.log.gz 2024-02-05T08:12:06.855 INFO:teuthology.orchestra.run.smithi043.stderr: 92.0% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-mon.a.log.gz 2024-02-05T08:12:12.445 INFO:teuthology.orchestra.run.smithi050.stderr: 93.8% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.5.log.gz 2024-02-05T08:12:12.542 INFO:teuthology.orchestra.run.smithi050.stderr: 93.8% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.6.log.gz 2024-02-05T08:12:12.602 INFO:teuthology.orchestra.run.smithi050.stderr: 93.8% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.7.log.gz 2024-02-05T08:12:13.002 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.2.log.gz 2024-02-05T08:12:13.022 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.0.log.gz 2024-02-05T08:12:13.559 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.1.log.gz 2024-02-05T08:12:13.611 INFO:teuthology.orchestra.run.smithi050.stderr: 94.0% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.4.log.gz 2024-02-05T08:12:13.613 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-02-05T08:12:13.614 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m9.373s 2024-02-05T08:12:13.614 INFO:teuthology.orchestra.run.smithi050.stderr:user 0m34.713s 2024-02-05T08:12:13.614 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m1.125s 2024-02-05T08:12:13.673 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/f211dd04-c3f7-11ee-95b6-87774f69a715/ceph-osd.3.log.gz 2024-02-05T08:12:13.676 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-02-05T08:12:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m9.438s 2024-02-05T08:12:13.677 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m39.903s 2024-02-05T08:12:13.677 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m1.301s 2024-02-05T08:12:13.677 INFO:tasks.cephadm:Archiving logs... 2024-02-05T08:12:13.677 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744/remote/smithi043/log 2024-02-05T08:12:13.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-05T08:12:19.027 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744/remote/smithi050/log 2024-02-05T08:12:19.031 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-05T08:12:24.042 INFO:tasks.cephadm:Removing cluster... 2024-02-05T08:12:24.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 --force 2024-02-05T08:12:24.240 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T08:12:24.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f211dd04-c3f7-11ee-95b6-87774f69a715 --force 2024-02-05T08:12:24.739 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: f211dd04-c3f7-11ee-95b6-87774f69a715 2024-02-05T08:12:25.045 INFO:tasks.cephadm:Removing cephadm ... 2024-02-05T08:12:25.045 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-05T08:12:25.061 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-05T08:12:25.077 INFO:tasks.cephadm:Teardown complete 2024-02-05T08:12:25.077 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-05T08:12:25.089 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-05T08:12:25.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-05T08:12:25.104 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-05T08:12:25.196 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-05T08:12:25.204 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-05T08:12:25.205 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T08:12:25.207 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-05T08:12:25.219 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-02-05T08:12:25.220 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-02-05T08:12:25.223 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T08:12:25.223 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-02-05T08:12:25.223 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +889us[ +881us] +/- 99ms 2024-02-05T08:12:25.223 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -788us[ -796us] +/- 57ms 2024-02-05T08:12:25.223 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +299us[ +292us] +/- 65ms 2024-02-05T08:12:25.223 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T08:12:25.224 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-05T08:12:25.225 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-02-05T08:12:25.225 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 88 +941us[ +935us] +/- 98ms 2024-02-05T08:12:25.225 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -782us[ -797us] +/- 57ms 2024-02-05T08:12:25.225 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +303us[ +288us] +/- 65ms 2024-02-05T08:12:25.225 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-05T08:12:25.225 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-05T08:12:25.233 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-05T08:12:25.234 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-05T08:12:25.244 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-05T08:12:25.345 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-05T08:12:25.424 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T08:12:25.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:12:25.455 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-05T08:12:25.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:12:25.486 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-05T08:12:25.496 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-05T08:12:25.504 INFO:teuthology.task.internal:Duration was 3129.881913 seconds 2024-02-05T08:12:25.505 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-05T08:12:25.513 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-05T08:12:25.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-05T08:12:25.515 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-05T08:12:25.556 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T08:12:25.569 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-05T08:12:25.858 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-05T08:12:25.858 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-02-05T08:12:25.858 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-05T08:12:25.914 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-02-05T08:12:25.914 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-05T08:12:25.938 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-05T08:12:25.938 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-05T08:12:25.956 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-05T08:12:26.007 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-05T08:12:26.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-05T08:12:26.024 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-05T08:12:26.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-05T08:12:26.514 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-05T08:12:26.514 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-05T08:12:26.541 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-05T08:12:26.568 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-05T08:12:26.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:12:26.584 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:12:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-02-05T08:12:26.633 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-02-05T08:12:26.654 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:12:26.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:12:26.676 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-05T08:12:26.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-05T08:12:26.708 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-05T08:12:26.721 INFO:teuthology.task.internal:Transferring archived files... 2024-02-05T08:12:26.722 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744/remote/smithi043 2024-02-05T08:12:26.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-05T08:12:26.788 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744/remote/smithi050 2024-02-05T08:12:26.788 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-05T08:12:26.827 INFO:teuthology.task.internal:Removing archive directory... 2024-02-05T08:12:26.827 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-05T08:12:26.829 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-05T08:12:26.875 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-05T08:12:26.887 INFO:teuthology.task.internal:Not uploading archives. 2024-02-05T08:12:26.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-05T08:12:26.939 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-05T08:12:26.940 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-05T08:12:26.942 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-05T08:12:26.953 INFO:teuthology.orchestra.run.smithi043.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 5 08:12 /home/ubuntu/cephtest 2024-02-05T08:12:26.956 INFO:teuthology.orchestra.run.smithi050.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 5 08:12 /home/ubuntu/cephtest 2024-02-05T08:12:26.957 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-05T08:12:26.967 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-05T08:12:26.986 INFO:teuthology.nuke:Checking targets against current locks 2024-02-05T08:12:27.012 DEBUG:teuthology.nuke:shortname: smithi043 2024-02-05T08:12:27.013 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T08:12:27.035 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:11:24.208051', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOwmtKAcalDVI3B6jBgDB9Ssf8LlUaBhXblimmLIHKW2lqRqQ0VfQPqu9I4AcQXsD3Cc3pKBj2YmYOssbWj+Zkk='} 2024-02-05T08:12:27.038 DEBUG:teuthology.nuke:shortname: smithi050 2024-02-05T08:12:27.039 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-05T08:12:27.062 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-05_05:57:01-orch:cephadm-wip-guits-testing-3-2024-02-04-2254-distro-default-smithi/7546744', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-05 07:11:24.206460', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeulAgSUwu4I3ZZhO4QiCggVVW3J4BWvQLe7wktvq7mTzLXwhcqyAMCGQRUb66gY1iNccN8AoH/srT7tkO9Noo='} 2024-02-05T08:12:27.084 INFO:teuthology.orchestra.console.smithi043:Power off 2024-02-05T08:12:27.085 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-05T08:12:27.109 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-02-05T08:12:27.109 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:12:27.122 INFO:teuthology.orchestra.console.smithi050:Power off 2024-02-05T08:12:27.122 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-05T08:12:27.131 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-02-05T08:12:27.144 DEBUG:teuthology.orchestra.console.smithi050:power off output: Chassis Power Control: Down/Off 2024-02-05T08:12:27.144 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:12:27.167 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-05T08:12:31.132 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:12:31.168 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:12:31.243 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-02-05T08:12:31.278 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is on 2024-02-05T08:12:35.245 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:12:35.279 DEBUG:teuthology.orchestra.console.smithi050:pexpect command: ipmitool -H smithi050.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-05T08:12:35.355 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-02-05T08:12:35.389 DEBUG:teuthology.orchestra.console.smithi050:check power output: Chassis Power is off 2024-02-05T08:12:35.457 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-02-05T08:12:35.490 INFO:teuthology.orchestra.console.smithi050:Power off completed 2024-02-05T08:12:35.597 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} duration: 3129.8819131851196 failure_reason: '"2024-02-05T07:40:00.000188+0000 mon.a (mon.0) 1944 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-02-05T08:12:35.597 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-05T08:12:35.772 INFO:teuthology.run:FAIL